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PREFACE 


The Flight Mechanics Panel (FMP) of the Advisory Group for Aerospace Research and Development has 
maintained a continuing involvement in assessing the fidelity of aircraft simulation. In keeping with this 
continuing interest and in recognition of the need for a common method of assessing motion system charac- 
teristics, the FMP, in October 1976, established Working Group 07 on “The Dynamic Characteristics of Flight 
Simulator Motion S ystems. " That Wo rking Group was char ged with defining and determining metrics of the 
hardware-delivered performance of the various degrees of freedom of the motidh“systemrits“report“(Ref~l)'" 
was issued in September 1979. 

As a follow-on activity to FMP WG-07, FMP WG-10 was established in March 1979 to consider "The Charac- 
teristics of Flight Simulator Visual Systems," This Working Group was charged with identifying and defining 
the physical parameters of the flight simulator visual display that characterize it and determine its fidel- 
ity. At its first meeting on 26-27 April 1979 at the British Ministry of Defense, London, the Working Group 
reviewed the terms of reference, defined in more detail its projected scope, and agreed upon three broad 
categories within which all of the parameters could be identified. At its second meeting at the Hochschule 
der Bundeswehr, Neubiberg, Germany, on 30-31 August 1979, the members reviewed the identification and defi- 
nitions of the engineering parameters, agreed upon a format for their presentations, and initiated the study 
of the importance of these parameters to the performance of a flight task and the degree to which this impor- 
tance is known. The third meeting of the Group took place on 21-23 April 1980 at the Ames Research Center, 
NASA, Moffett Field, California. At that meeting, the results of the study efforts of the members were dis- 
cussed and put into formal context as definitive characteristics. Problems of measurement were discussed 
and alternatives were reviewed. Tentative agreements were reached on statements regarding the relative 
importance of the parameters. All members were asked to consider the unknowns in this area and to recommend, 
prior to the next meeting, the research needed. At its final meeting on 24-26 September 1980 at AGARD Head- 
quarters, Paris, the draft of the Group's final report was reviewed and approved. 
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1. INTRODUCTION 

The importance and usefulness of out-of-the-window visual simulation displays can be judged best by 
the recent large increase in the use of this equipment in air-carrier trainer simulators. The military has 
followed suit in that the visual systems for flight simulators have become a major portion of the large 
simulator budget of the U.S. Air Force, especially since the feasibility and flexibility of computer- 
generated image systems have been demonstrated. 

._Equipment„for _out-o.f-_the-window visual Simula tion_was_ldrgely_developed--in the-1960's-by^aircraft 

manufacturers who used the simulators for engineering and test purposes. By the end of that decade, the 
equipment had proved sufficiently useful and dependable that it was being adopted for training. At the end 
of the 1970' s, over 300 visual simulation systems were in use by the world's air carriers. The relatively 
late addition of these visual cues to training simulators, compared with inside-the-cockpit visual cues and 
with motion cues, may be ascribed to several factors. One of these factors is the recent rapid development 
in the techniques and hardware of computers and television which have greatly increased the quality and 
reliability and have decreased the cost of the visual simulation equipment. Another i s the success of early 
users of visual simulation equipment in decreasing the cost of pilot training by substituting simulator time 
for training time in aircraft. 

Perhaps the principal reason for recent acquisitions of simulator visual systems for combat aircraft is 
their potential for improving combat readiness training. Modern weapon system simulators, with their sophis- 
ticated visual systems, provide a means by which flight crew members can experience maneuvers or situations 
that in reality would be either too dangerous to risk in training or are such that they would only be encoun- 
tered in actual combat. Moreover, using these systems can conserve fuel and still provide adequate training 
of military pilots for the multitude of different tasks that they might face. Furthermore, it is evident 
that the more accurately the requirements for, and the effects of, various aircraft design concepts can be 
predicted, the more cost effective the total system becomes. 

Visual simulation for air carrier use has largely been restricted to a forward-looking view of the air- 
strip and its surroundings during takeoff, approach, and landing. The conventionally used 48° horizontal by 
36° vertical field of view has proved satisfactory for training pilots for a straight-in approach of an air- 
craft; regulatory agencies have certified systems of this type as capable of substituting for a large propor- 
tion of aircraft training time. However, military training use has lagged because of the more varied visual 
simulation requirements associated with military missions. The tasks performed by aviators in the modern 
battlefield environment are many and diverse. They range from takeoffs and landings at airports with all 
the normal landing aids to operations out of hastily prepared landing strips in unfamiliar terrain; from 
air-to-air combat to nap-of-the-Earth missions; from in-flight refueling to tank busting. These military 
tasks impose more severe requirements on the out-of-the-window visual simulation system and make the hardware 
solutions much more difficult. It is not practical to design a simulator that can do all the tasks or 
missions associated with a particular aircraft. It may not even be possible. Current philosophy seems to 
favor a variety of part-task trainers, each having the essential elements optimized for the particular task 
or set of tasks. The simulator system that is mainly responsible for this state of affairs is the visual 
system. Although aviators have a variety of electronic aids to assist them, many missions still require the 
human visual system and the intimate knowledge that system gives the aviator of his external environment. 

Out-of-the-window visual simulation is a formidable chal lenge. because of the fantastic performance capa- 
bilities of the human eye. The human visual system provides the user with most of his sensory input: the 

peripheral retina answers the "where" question and the central foveal area the "what" of pattern vision 
(Ref. 2). The eyes are our most important sense organs for gaining information about the world around us. It 
is estimated that over 90% of the information that we receive during our normal daily activities comes through 
the eyes and certainly that much or more when an aviator is involved in flight tasks. 

Man's eyes have variable focus and aperture control. The maximum acuity of the human eye occurs only in 
the cone of foveal vision (a cone of about 3°) but this cone is mounted in movable eyeballs, the eyeballs are 
in a movable head, and the head is on a movable body. These characteristics of the human visual system make 
the initial requirements of any simulator visual system to be the foveal vision capability over a 4 n-sr 
field of view. Moreover, man can bring to bear over this large field of view a visual resolution capability 
of about 1 aremin. 

It is, of course, totally impractical to set the requirements of a visual simulation system to match the 
performance of human eyes. A few calculations will convince the visual simulation system designer that 
designs to meet the human eye characteristics will require a system performance exceeding by 10 to 100 times 
the performance of systems currently used by air carriers. The requirement for a complete reproduction of 
the crew's available visual stimuli will be compromised as surely as would a proposed requirement that the 
pilot’s available motion stimuli be totally duplicated. The objective is simulation of the visual world; it 
is not duplication of that world. 

It is intuitively understandable that to duplicate the pilot's motion cues would require a simulator 
motion system physically approaching the volume through which the actual aircraft would fly, and that, there- 
fore, duplication of motion cues is neither technologically nor economically feasible. It is less obvious 
but equally true that duplication of the pilot's available visual stimuli in a simulated mission is not 
currently technologically feasible. If it were technologically feasible, full duplication would be employed 
in a brute force approach only because the understanding of the visual stimuli necessary for a given simu- 
lated mission are not well understood. One would duplicate them because one had no idea how to simulate 
them economically. 

Those who set requirements and determine specifications for simulator visual systems face a problem. 

The technology is inadequate to allow them to provide the "ideal" system and our understanding how a human 
uses the visual information in a. simulator is insufficient to provide clear guidelines on how to make the 
necessary tradeoffs. However, the increasingly successful use of out-of-the-window visual .simulation equip- 
ment is stimulating the development of better equipment as well as the research to provide data and insights 
into system requirements. The growing importance of simulators as training devices has led to the design 
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and operation of simulators for research in training techniques and for studies of the relative importance 
of the several visual system characteristics. The increased interest in the field has attracted the atten- 
tion of psychologists and other scientists interested in human performance and behavior, and they are con- 
ducting studies, using simulators and other laboratory apparatus. As a result of these activities, there 
is an increasing body of information available that is directly applicable to the problem of making tradeoff 
decisions from aircraft mission requirements into human performance requirements and finally into simulator 
engineering specifications. 

The literature on visual perception provides information relevant to some aspects of the visual simula- 
tion system design. However, out-of-the-window scenes, with their requirements for large fields of view, 
special optics, real-time dynamics, complete feedback, and special effects, impose requirements that are not 
addressed in the literature. It is unlikely that a complete understanding of visual perception will be 
obtained in the near future. Without complete knowledge of the relationship between the physical and the 
behavioral continuum, we have often fallen back on the philosophy of duplicating everything that might be 
important. As stated previously, we cannot afford this approach - if we could, simulation would be unneces- 
sary. The continuing task, then, is to define the design characteristics that might affect perception of 
physiological responses, to establish the relative importance of the corresponding visual and physiological 
effects, and to understand their relationship with the physical continuums of the displays that we can now 
generate. 

This report addresses only a very small part of the total problem. The objective was to identify and 
define the physical parameters of the flight-simulator visual system that characterize the system and deter- 
mine its fidelity. The desire was to establish the physical measures of image quality that are describable 
in objective terms. It is convenient to discuss visual simulation system characteristics in terms of the 
three basic categories of spatial, energy, and temporal properties corresponding to the three fundamental 
quantities: length, mass, and time. However, that these measures of the visual simulation system are not 

totally independent is made obvious throughout the discussions of this report. The Group also could not 
address these objective measures without considering the psychological aspects of visual perception. As one 
member of the Group said: "Describing a picture in engineering tenns is similar to describing a book by the 
quality and quantity of the print and paper," Nevertheless, this report discusses the characteristics of 
visual simulation systems within the categories of spatial properties (Sec. 2), energy properties (Sec. 3), 
and temporal properties (Sec. 4). For each parameter related to these categories there is a description of 
its effect, a definition of its appropriate units or descriptors, a discussion of methods of measurement, 
and a discussion of its use or importance to image quality. The Group intentionally avoided trying to 
establish recommended values for the characteristics, for such values are likely to depend on the tasks for 
which the simulator will be used. 

Unfortunately, we cannot claim complete understanding of the relationships between these objective 
factors and the psychophysical factors of visual perception. As a critical part of any man-machine system, 
the human operator receives information, processes it, and takes some specific action upon the system. 
Although we may think of such input information in appropriate physical units and physically measure it by 
such units, it is unfortunately true that the human sensory system does not usually encode and perceive the 
information in the same physical units or with the same resolution, range, or fidelity. The relationship 
between the physical dimensions of an input and the observer's perception of that input is the rightful study 
of psychophysics. Thus, we have the psychophysical scale of hue as a psychological correlate of dominant 
wavelength and the psychophysical scale of brightness as a correlate of the luminance of a specified visual 
stimulus. Mean luminance, field of view, viewing distance, contrast, scene movement, signal-to-noise level, 
aspect ratio, resolution, and target and background characteristics are all known to have significant 
(although often inconsistent) effects upon operator information-extraction performance. Individual experi- 
ments have tended to examine the effects of one, two, or sometimes three such variables. However, due to 
the inherent interaction among these variables in their effects upon observer performance, quantitative com- 
bination of the results is hazardous even in the presence of good experimental control and measurement. 
Therefore, recent efforts have been oriented toward the development of analytical expressions of overall 
image quality, such as the modulation transfer function area (MTFA) (see Sec. 3.4). One is often concerned 
with selecting the best imaging system for a broad range of applications, with the objects to be recognized 
undefined, the viewing conditions only loosely specified, the environmental considerations unpredictable, 
and the future possible uses of the system perhaps totally unknown. In such an application, one is inter- 
ested in determining the performance to be expected from each of several candidate systems, any of which 
would be used for a myriad of purposes. . For this application, the MTFA is a metric of overall image quality, 
one that appears to predict operator performance in recognizing various objects of interest. 

In Section 5, we have attempted to address the deficiency in our knowledge of the relations between 
visual perception and the characteristics of the visual simulation system by documenting the experience of 
the Working Group members regarding the importance of these parameters in accomplishing a given visual task 
under given conditions. The impressive array of elements available for visual simulation systems would seem 
to make the design of a system a straightforward engineering process. However, despite rapid changes in the 
technological feasibility, particularly in computer image generation, it is still not possible to assemble 
the perfect system usable for all aircraft and all missions. Tradeoff decisions will have to continue to 
be made to provide a solution that is technically and economically feasible. 

Two kinds of information are needed to make these decisions. The first is the knowledge of what visual 
stimuli are used by pilots as cues when flying aircraft in specific missions or tasks. This may be obtained 
subjectively from pilots or from psychophysical experiments. The second kind of information that is needed 
is the specification of the characteristics of the visual simulation system that make it possible to present 
these cues to the pilot with sufficient fidelity to allow the given tasks to be performed. What do we need 
to fool the simulator pilot? For example, there is evidence that even if the characteristics of the visual 
system are not as good as the characteristics of the human eye, one can change the picture content to make 
the performance of the operator the same as it would be in the real world. These questions have been 
addressed, in part, by the AGARD's joint Working Group (of the Aerospace Medical and Flight Mechanics Panels) 
on fidelity requirements of simulation for training purposes (see Ref. 3). 

The final sections of this report present projections of future trends, recommendations for research, 
and concluding remarks. 
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2. SPATIAL PROPERTIES 

2.1 Introduction 

The spatial properties of a flight-simulator visual system are those that relate to the spread of a 
visible image over space. There are boundaries of the image termed the field-of-view and limited space 
called the viewing region from which the image may be seen if the viewer^s eye is contained within it. 
Furthermore, the image is located at a definite distance from the observer's eye leading to depth considera - 
tions . As a consequence of imperfections in the visual system, an image detail may not appear to lie at its 
intended location in space, giving rise to errors in location called mapping factors . Finally, the displayed 
image itself is composed of a myriad of observable light patterns that comprise the scene content . 

Since the simulated aircraft is dynamic, that is, it translates through space and changes its attitude, 
the properties just mentioned are also dynamic, especially considering that the observer may change his view- 
ing position, not only because of aircraft movements, but also because of his own head and eye movements. 
Indeed, advantage is taken of this fact by simulator designers so that only the portions of the "simulated 
world" that need to be seen at any particular instant are displayed. The windows of the simulated aircraft 
form a spatial limit to the observable outside world and, generally, the field coverage is never made any 
larger. The problem is usually to make the simulated coverage as large or "large enough." 

The spatial properties result from the geometrical array and size of the display elements, the parts 
that actually "write the image," and the optical relays (mirrors, lenses, screens, etc.) that convey the 
light to the observer's eye. However, the scene content properties are associated with the modulation of 
the light spread over space that results in an interpretable image. These properties are not only dependent 
on the display elements, but also on the image-generation performance. 

This section deals with the spread of light sources over space, and the transmission of their light to 
the observer* s eye. 

2.2 Field of View 
2.2.1 Definition 

Field of view is a commonly used term that describes the extent or coverage of a flight-simulator visual 
system. A picture is displayed within the field-of-view boundary and generally darkness is evident outside 
it. A simple window creates a field of view within it. There are three fundamental characteristics of a 
field of view: shape, size, and orientation. 

One horizontal and one vertical angle are often used to describe the shape and size of a field of view. 
In the case of small fields of view located near the horizontal, there is usually no confusion of meaning. 
However, when the field of vie\; is large or when parts of it are far from the horizontal, the use of two 
plane angles as descriptors can give deceptive impressions of size and shape. In an analogous situation, the 
Mercator projection gives distorted impressions of the size and shape of regions near the poles. 

The shape of a field of view may be defined by describing points on its boundary in terms of two plane 
angles referenced to a convenient origin, direction, and order of rotation. The origin of the axis system 
is placed at the observer's eye, and the reference direction is usually that of the longitudinal aircraft 
body axis. The coordinates of a point on the field boundary are determined in the following way: the look 

direction is first rotated toward the right wing tip about an axis normal to the reference direction and then 
rotated toward the aircraft-referenced zenith until it intersects the point in question. The angle of the 
first rotation is called the azimuth angle of the point; the second is called the elevation angle of the 
point. In the same way, the location of other points on the field-of-view boundary may be determined by 
noting the successive angles in which the look direction must be similarly rotated in order to align with 
each point. A plot containing the azimuth and elevation angles of many points on a field boundary consti- 
tutes a field-of-view plot and may be compared to a similar plot for the windows of an aircraft or used to 
determine whether a particular object outside the aircraft is occluded. 

The size of a field of view is described by the solid angle. This is a measure of the angular spread 

at the vertex of a cone (or similar figures), measured by the area intercepted by the conical surface on a 

unit sphere with its center at the cone vertex. The solid angle of a simulator field of view is found by 
placing. the center of a sphere at the observer's eyepoint, measuring the area in this spherical surface cut 
out by a radial line sweeping the field-of-view boundary, and dividing this area by the square of the radius 
of the sphere. 

The unit of measurement for solid angles is the steradian (sr). This is the solid angle that inter- 
cepts an area on the surface of the sphere equal to the square of the sphere's radius. The solid angle that 

cuts out a full sphere is equal to 4 n sr. Thus, a full field of view (the maximum field size possible) 
equates to 4 ir sr. The size of any other field may be expressed as a fraction of the full field. 

In addition to the dimensionless steradian, other dimensional units are used for measuring solid angles; 
for example, the square degree used by astronomers, or the square grad. The measures of a full field in 
various solid angle units are given in Table 1. 


SOLID ANGLE VALUE FOR A FULL FIELD 

UNITS 

4 IT 

Steradians (square radians) 

41,253 

Square degrees 

50,930 

Square grads 


TABLE 1: Solid angle value for a full field-of-view 
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The field-of-vlew size is useful in some cases in determining the brightness of displays, and may have 
application in the documentation of scene content in terms of displayed object density, that is, the number 
of objects displayed over a given solid angle. 

The orientation of a field-of-view boundary has already been defined in terms of azimuth and elevation 
angles relative to body axes fixed to the simulated aircraft. In recent times, attention has been given to 
area-of- interest displays. These are displays in which the field-of-view boundaries are no longer fixed to 
the aircraft but move relative to it. The two most common examples are displays in which the field-of-view 
center is directed either to the point in the simulated world toward which the viewer's head is pointed, or 
to the location of an object of interest, such as a target; hence, the term "area-of-interest." The area- 
of-interest display, may have a relatively small field of view that is slaved to the direction of the 
observer's head or eyes or to an object in the simulated world. By virtue of this slaving of the boundary, 
a much larger field coverage is achieved, the coverage being the maximum envelope of the moving instantaneous 
field boundary. 

To describe fully the field coverage of an area-of-interest display, it is necessary to define not only 
the field of view of the area-of-interest display but also the region over which this field of view can be 
driven. The boundary descriptors previously mentioned are used, and three additional angles describe the 
orientation of the area-of-interest field of view relative to the cockpit. Again, two plane angles to a 
field reference point are defined in the conventional order of azimuth, then elevation, and a third angle 
must be included that describes the roll angle of the field boundary about the straight line from the view- 
ing point to the field reference point. 

2.2.2 Measurement methods and illustrations 

Theodolites and transits are commonly used to measure angles to objects relative to an arbitrary coor- 
dinate frame whose origin lies at the intersection of the instrument's gimbal axes. The order of rotation 
is generally consistent, that is, the^same as the order described earlier and, therefore, large angles as 
defined here may be read directly from the instrument. A reticle is generally provided to facilitate align- 
ment. Care must be taken to obtain an instrument capable of focusing at the close range of real -image dis- 
plays, at the edges of virtual -image displays, or at close-by cockpit structures. 

The angular dimensions of the field of view of a flight-simulator visual system can range from a few 
degrees to angles equivalent to those of a full field. Commonly used TV camera lenses and commercial 
airline- type displays will produce scenes having field boundaries that subtend angles of, at most, about 50° 
horizontally and 35° vertically; the field size is approximately 4% of full field. A telephoto camera lens 
will reduce these angles. By contrast, the U.S. Air Force's Advanced Simulator for Pilot Training (ASPT)^ 
(Ref. 4) has an irregular field boundary shape with corresponding azimuth angles extending to ±150° and ele- 
vation angles of as much as +110° and -40°. Its field size is approximately 58% of full field. 

The binocular field size of the human (with head and eyes fixed) is similar to that for the ASPT; how- 

ever, the field shape is not. The field angles extend to approximately ±90° in azimuth and +50° and -80° 
in elevation. An area-of-interest display slaved to a seated observer's head naturally would not require a 
field size of more than 50% of full field. However, the field coverage would be nearly the full field due 
to the large head rotations possible. These are generally taken to be ±70° in azimuth, ±50° in elevation, 
and ±45° roll for comfortable viewing. With upper torso movements, azimuth angles of ±150° can be reached. 

Other field illustrations will be Instructive here (see Fig. 1). For example, consider the circular 
field shape (Fig. 1(a)) where the semi diameter of the boundary subtends an angle e from the viewpoint. 

The field size is found by integration to be the following: 

Field size of a circular boundary = 2 tt( 1 - cos e) sr 

Using a similar method for a zone (Fig. 1(b)), a shape similar to the surface of the Earth contained 
between parallels of latitude and meridians of longitude, yields the following: 

Field size of a zone = 2 ^2) 

where 'i' is the angle between the bounding meridians and 61 and 62 are the angles equivalent to the north 
and south bounding latitudes, respectively. This example illustrates the difference between a wide-angle 
field and a large field size. A zone extending to large horizontal angles (v 360°) with a small height 
(0j, 02 0) represents a wide-angle field of small size. 

The above examples are plotted in Figure 1. The field sizes are shown as functions of total azimuth 
and elevation angles for a circular shape and a zone. Several other field-of-view boundaries, including 
that for the ASPT, are plotted in Figure 2. 

It may be noted that the field plots in Figure 2 show highly distorted shapes for the pentagonal window 
elements of the ASPT field. This is particularly true of the overhead window. Its shape is distorted as a 
result of trying to represent a three-dimensional surface on a two-dimensional plot. A similar problem is. 


^The Advanced Simulator for Undergraduate Pilot Training (ASUPT), now called ASPT, was developed during 
the early 1970* s as an advanced simulator to support flying training research in undergraduate pilot train- 
ing. It is a two-simulator system, originally designed to simulate the T-37B, the U.S. Air Force's primary 
jet trainer, which has a side-by-side seating arrangement. The visual display, a mosaicked in-line, infinity 
system, was positioned about the left-hand or pilot's seat and oriented to best fill the T-37B field-of-view 
with its seven pentagon-shaped channels. The visual system imagery is generated via a multichannel computer 
image generator. Although the mathematical models and cockpits have been changed to represent an A-10 and an 
F-16 aircraft, the visual system remains virtually unchanged. The ASPT is operated by the U.S. Air Force 
Human Resources Laboratory, Williams Air Force Base, Arizona. 
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Fig. 2 Typical field plots 


of course, encountered in cartography when attempts are made to map the surface of the world; many methods 
of accomplishing this surface mapping have been advanced by cartographers. For a detailed analysis of the 
subject, the reader is referred to works on cartography (e.g.. Ref. 5). 

There are several map projections in wide use that attempt to depict, without large distortions, fea- 
tures on the globe using a plane surface. Some of these are referred to as “equal -area" projections; equal 
areas on the map represent equal areas on the globe, although shapes are somewhat distorted. The use of 
equal -area projections to plot field-of-view boundaries is recommended^ when the field size is large, that 
Is, greater than 50X of a full field. For smaller fields, a linear plot such as the one shown in Figure 2 
1s sufficient. Figure 3 is a repeat of the plots of Figure 2 on a Hammer equal -area projection. Appendix A 
contains a sample Hammer field-of-view grid and additional examples of typical aircraft and simulator field- 
of-view plots. 


HUMAN 



Fig. 3 Typical field plots on Hanmer's equal -area 
projection of a sphere 

2.2.3 Importance of field of view 


Obviously, a way to describe a large field-of- 
view is to scribe its boundaries on a sphere. This 
can be accomplished using a transparent sphere and a 
rod with one end fixed at the center of the sphere. 

By mounting the sphere with its center at the intended 
view point and sighting along the rod to the position 
of any object or boundary in the field of view, the 
locations of these objects or boundaries can be marked 
on the sphere with a grease pencil. In this way, real- 
world field boundaries, such as those provided by cock- 
pits, may be readily documented and compared with 
simulator fields of view. 

The transparent sphere method is a simple, inexpen- 
sive, and convenient way of documenting field-of-view 
boundaries; however, its accuracy cannot be expected to 
be better than to within a degree or two. In cases 
where greater accuracy is needed, surveying instruments 
must be used. 


The field of view is the pilot's window to the world. Because certain parts of the outside world scene 
are important to the accomplishment of the flight task, the shape, orientation, and size of the field of view 
can greatly influence the way that a pilot flies the simulated aircraft. Certain aspects of the visual scene 
can stimulate the perception of orientation and movenent more easily than others. Also, the detection and 
recognition of objects may be an important part of the pilot's task, such as in air-to-air combat and air-to- 
ground weapon delivery. If any of the visual scene features important to the task are outside the simulated 
field boundary, the pilot's performance and behavior will likely be different from what it would be in the 
real world. Although it is difficult to generalize about field-of-view importance, a large field is more 


^The projection of Hanmer (also known as Aitoff's projection) is slightly preferred, but those of 
Sanson-Flamsteed, Mollweide, and Lambert are also acceptable. The equal-area projection most conmonly 
found in the simulation literature is that of Hammer. 
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desirable than a small one. The sensation of reality induced by a visual wide- field display was confirmed 
in the recent experiments of Hatada et al. (Ref. 6). 

With computer image generation has come the ability to produce multi window simulators; the usual dis- 
play medium is the color television display, collimated and covering a 36® by 48® field of view in each 
window. A careful distribution of such displays is called for, and their locations are task-dependent. 

For civil applications, three such displays have been acceptable, but the high turn rates and large bank 
angles used by pilots of military aircraft suggest that more than three are essential, if there is to be 
a useful extension over the training achieved from single-display systems. To achieve full coverage of the 
view from a typical fighter cockpit, however, at least 16 of these television display devices would be 
needed, and severe design problems (such as access to the cockpit) would arise. 

Although a large field of view may be desirable, it is not possible, using current technology, to 
achieve a high-resolution display of a highly detailed scene over a very large field. Consequently, 
designers and users of simulator visual systems are being forced to consider the area-of-interest display 
concept. This concept is based on the assumption that only certain portions of the outside world need be 
displayed with high resolution and detail at any instant and that the rest of the visual field may be 
greatly degraded in detail and resolution. The imagery representing each of these portions can be manip- 
ulated in response to head or eye movement or by target location according to whether it will be seen in 
the fovea 1 or peripheral regions of the eye. 

In a common method of providing a wide field of view, a large spherical screen, enclosing the cockpit, 
on which there are images from both television projectors and point-light-source/transparency-projection 
devices, is used. A point-light-source device, such as a sky/ground projector, does not produce a high- 
resolution image, but it covers a very wide unbroken field of view, and the image is. satisfactory for visual 
attitude reference. A technique of insetting a small high-resolution television scene in this field is an 
attempt to match the simulator visual system resolution to the performance of the eye, which is greatest 
only in the relatively small foveal area. Optimum use of this technique requires information concerning 
the line of sight of the eye in order to position appropriately the high-resolution part of the scene at 
which the pilot is looking. 

Hatada et al. (Ref. 6) have identified the visual fields that might provide the basis for designing 
the field of view of an area-of-interest display. The discriminatory human visual field is within the 
central 3® cone of foveal vision within which the observer has the highest discriminatory capability and 
visual acuity. Outside of this is the effective visual field, which extends to about 30® horizontally, 

8° upward, and 12® downward with eyes and head fixed. In this region, the visual acuity falls to about 
one-fifth that in the discriminatory field, but the observer can bring the discriminatory-field ability 
to bear with eye movement only. The induced visual field is outside the effective visual field, extending 
to about 100° horizontally and to about 85® vertically; it is the region within which the observer can 
recognize the existence of a visual stimulus. Finally, the supplementary visual field extends to about 
200° horizontally and 125® vertically. This region has no direct functional role in the perception of 
visual information, but it can arouse a shift of the observer's gaze in response to abrupt stimuli. 

An area-of-interest visual display would take advantage of these characteristics. A possible design 
of such a display would include an area of high resolution and high detail of the size of the discriminatory 
and the effective fields, blended into a display for the peripheral region with characteristics correspond- 
ing to the induced and supplementary field, and driven by the observer's head motion. 

. The attractiveness of the area-of-interest display lies in its potential for providing a large field 
coverage while requiring only a modest image generator capable of "filling" only the instantaneous field 
of view. For a head-slaved area-of-interest display, the instantaneous field size need not be more than 
half the full field value of a human because this is approximately the field size of the human with head 
fixed. For eye-slaved system, the potential exists for further reductions in the required scene content 
because of the differences between the characteristics of human foveal and peripheral vision. Although the 
various area-of-interest display concepts offer potential savings in hardware complexity, they also pose 
potential dynamics problems because of the slewing speeds possible. The head is capable of rotating much 
faster than an aircraft, and the eyes faster still. Inaccuracies in head or eye tracking, or lags in the 
servosystem, may result in objectionable visual illusions or unusable dynamic imagery. In addition, little 
data exist on the dynamic acuity of the human visual system, and care must be taken in the design of area- 
of-interest systems to insure that Important cues outside the area of interest are appropriately presented 
to the pilot or vehicle operator. 

2.3 Viewing Region 

2.3.1 Definition 

All visual simulation systems have the limitation that they can produce good imagery only when viewed 
from within a specific region of space. This viewing region consists of the region of space within which an 
acceptable image may be received by the eye; when the visual system is viewed from a point beyond the bound- 
ary of this region, the imagery disappears or its quality is unacceptable. This effect is due simply to the 
geometrical relationship between the viewing point and the optical elements. In the case of coninon binoculars, 
the viewing region is termed the "exit pupil" and is a small cylindrically- shaped space a few millimeters 
outside each eyepiece. It is usually less than 10 mm in diameter and 5 nm thick, with its axis coincident 
with that of the instrument's optical axis. When the iris of the eye is placed in this region, the best 
imagery with the largest field-of-view can be seen. In the case of cinema, the viewing region is substantially 
larger. Nevertheless, as the observer moves away from the best viewpoint (for example, the center of a 
spherical wrap-around screen), the Images become skewed (distorted) and the brightness of various regions of 
the screen may change. 
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2.3.2 Measurement methods and illustrations 


The measurement of the viewing region of a visual simulation system involves nothing more than the mea- 
surement of any performance characteristic and the definition of the region boundary by the limit of accept- 
able value of that parameter. The same apparatus that is used to measure the characteristic in question at 
the center of the viewing region is moved and new measurements made. The boundary of the viewing region is 
defined by those surveyed locations at which minimally acceptable performance is measured. Any convenient 
orthogonal coordinate frame may be used with any linear dimensions and units to define the viewing region. 
The description of a specific shape, for example, a hemispherical, spherical, or cylindrical volume, can 
also be used to describe the region. 


The viewing region for a circular cathode ray tube (CRT) display device with spherical mirror-beam- 
splitter collimating optics is illustrated in Figure 4. In this example, the CRT face is assumed to lie 

in the spherical focal surface of the mirror-beam- 
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Fig. 4 Spherical mirror beam-splitter viewing region. 


splitter combination, that is, in the spherical 
surface whose radius is half that of the mirror. 

Radial rays leaving this surface will, after reflect- 
ing from the top surface of the beam splitter and the 
right surface of the mirror, pass through the beam 
splitter and emerge nearly parallel. Theoretically, 
this arrangement should produce a conical high-quality 
viewing region as shown. When viewed from outside 
this conical region, a vignetting effect occurs in 
which only a portion of the CRT face is visible. 

The high-quality viewing region for the ASPT 
is a forward- facing hemisphere of 15.0 cm radius with 
its center at the intersection of the optical axes of 
all seven in-line pentagonal windows. By contrast, 
the U.S. Air Force's Multiviewer Project seeks to 
create a collimated display with a viewing region of 
about 1. 5x1x0. 5 m. The Multiviewer concept (Fig. 5) 
embodies a large collimating mirror and a rear-screen 
projection display. By virtue of its size, it has a 
large viewing region usable by both the pilot and 
copilot of a transport aircraft. 


IMAGE PROJECTORS 



Fig. 5 Multi viewer display showing viewing volume 
(adapted from Ref. 2) 


2.3.3 Importance of viewing region 

The importance of a viewing region lies in its 
size compared with the envelope of expected aircrew 
viewpoints. The normal head movements of pilots 
while flying, coupled with the normal distance between 
the eyes, dictates that the minimum viewing region be 
of the order of a forward-facing hemisphere with a 
radius of at least 15 cm; in certain applications, 
for example, in air-to-air combat, the required view- 
ing region could be considerably greater. Certainly, 
if more than one crew member must see the image, the 
required viewing region must either be larger or be 
duplicated for each observer. 

The image viewed outside the viewing region of 
collimated displays is usually so highly distorted or 
occluded as to be unusable. However, for real -image 
displays, that is, screens and projectors, the degra- 
dation is usually so gradual that the imagery can 
still be useful at viewpoints considerably distant 
from the optimum location. This latter factor often 
is the reason for using real -image displays for 
multiple observers. 


2.4 Depth Considerations 


2.4.1 Definitions 

There are three factors important to a discussion of depth as it pertains to a visual simulation system: 


1. Real-world range: This is simply the true distance from the observer's eyepoint to an object or 

point in the real-world scene. 

2. Actual range: This is the range of the image of that object or point in the visual simulation system 

as would be measured by an optical rangefinder. 


3. Range effects: In any visual simulation system, images of objects at various ranges are susceptible 
to variations in quality. Some variations are desirable. For example, the familiar reduction in visibility 
with Increasing range is a common natural phenomenon. However, a reduction in the apparent fine detail asso- 
ciated with close objects is not and this occurs in visual simulation systems due to an optical property of 
lenses called the "depth-of-field." In all optical systems, a range of distances exists within which objects 
are imaged onto the film or viewer with a specified quality. Outside this range, termed the "depth-of-field," 
the quality is reduced. 
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2.4.2 Measurement methods and illustrations 


Any convenient unit of length may be used to describe the real-world or actual ranges. In cases in 
which collimators are used to display an image at a large distance, the collimating performance may be 
described in terms of an angular deviation e of a light ray from a reference direction in units of mils 
or mil 11 radians. This is illustrated in Figure 6. The deviation may be outward, in which case the angle 
is called divergence and corresponds to an image that would be located, by using tri angulation, on the dis- 
play side of the observer. When the ray deviation is Inward, it is termed convergence, and, correspondingly, 
the image would be found behind the observer. 


There is still another angular deviation of a 
light ray that is called dipvergence. In this case, 
the deviation is normal to the plane containing the 
image and the two lookpoints. For example, a 
reference ray may emanate from the image point and 
pass through the first lookpoint. A short distance 
away, at the second lookpoint, a second ray is 
sampled and found not to lie in the plane contain- 
ing the first image and the two lookpoints. This 
is dipvergence. 

The distance of an image formed by a colli- 
mator may be derived from the convergence or 
divergence by dividing the lookpoint separation 
distance by the deviation in radians. 
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Fig. 6 Collimating descriptors 


Another descriptor of distance, especially useful for large distances, is the diopter. This is simply 
the reciprocal of the distance expressed in meters. 


A coimion rangefinder may be used to measure the distance between the observation point and the image of 
a particular object in the scene. If the separation of the apertures of the device is equal to the mean 
interpupillary distance (about 70 mm), the measurement can be related to human visual performance. Since, 
as we have seen above, some collimators place the apparent image behind the observer (in other words, on 
the side of the observer opposite to the display), it is necessary that the rangefinder by able to measure 
converging, as well as diverging, rays. This is equivalent to a "toeing out" of the optical axes of the 
instrument. To measure dipvergence, the rangefinder must be capable of rotating the sight direction of 
one of Its elements out of the plane containing the reference image and the first and second look^points. 

The depth-of-field boundaries for the camera probe of a typical camera-model simulator visual system 
are shown in Figure 7. The lines represent values of constant limiting resolution. In this figure, the 
straight line labeled "probe diffraction limit" defines the resolution limits due to diffraction effects. 

The probe optics are adjusted to produce the best image of objects at this range. At longer and shorter 
ranges, the resolution, as the graph shows, is degraded. 


Figure 8 shows the typical off-axis collimating performance of the spherical mirror/beam-splitter 
collimator shown in Figure 4. In Figure 8, the range (expressed in diopters) is shown for an image at the 
center of the CRT face as it is viewed from points at various distances off-axis at two different x loca- 
tions. A perfect collimator will place the image of the CRT center at optical infinity, corresponding to 
zero diopters. In this case, negative diopters mean that the image is actually on the opposite side of the 
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Fig. 8 Off-axis collimating performance of a spherical 
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observer from the beam splitter; that is, the emerging rays converge slightly. The fact that the mirror/CRT 
does not precisely collimate over its high-quality viewing region may be due to aberrations in the mirror, 
in the tube face or monitor, or in both. 

In the case of an image produced by projection onto a screen, the actual range is, of course, the dis- 
tance of the observer from the screen element containing the image. 

2.4.3 Importance of depth considerations 

Depth perception is that property of vision that results in the world being perceived as three-dimensional 
rather than flat. There are many mechanisms by which this takes place but not all can be simulated effec- 
tively. The mechanisms can be separated into two groups: those associated with binocular vision (use of 

both eyes simultaneously) and those associated with monocular vision (one eye only). 

The viewing point of each eye is different (binocular effects^) and, therefore, the corresponding 
retinal images are also slightly different. Physiologists have suggested that the two images formed by the 
eyes are fused in the visual cortex and that visual pathways starting from nerves at corresponding points in 
each retina are associated in the cortex so that disparate images can be used to form the impression of depth. 
This is called binocular disparity . The impression created is one of a three-dimensional version of the 
objects in the vTsual field, as would be seen by a single eye located midway between the eyes. According 
to Reference 7, the most accurate mechanism for perceiving depth, using internally-generated cues, is the 
stereopsis resulting from this binocular disparity. Experimentally-determined values of stereoscopic acuity 
(smallest detectable angular disparity) are in the range of 10 to 2 arcsec. The reference states, for 
example, that an observer with stereoscopic acuity of 5 arcsec can discriminate that an object at 2,600 m 
is closer than one at infinity. 

A further binocular cue of range is produced by the slight *'toeing-in" of the eyes when looking at a 
near object; this mechanism is called convergence and provides a weak cue at ranges up to about 20 m 
(Ref. 7). 

There are quite a few other clues of depth and range that do not require both eyes (monocular effects). 

For example: 

1. The slight change in accommodation (focus) for close objects is effective at ranges less than 
1 m (Ref. 7). 

2. Motion parallax describes a change in relative position of objects due to either object or observer 
motion. 

3. Apparent size of familiar objects is also a powerful cue. 

4. Linear perspective or the geometrical aspect of a vanishing point, parallel or convergent lines, and 
related effects are important. 

5. Interposition or the masking of objects by others that are closer. 

6. Aerial perspective is the loss in contrast produced by atmospheric effects at increasing distance 
from the observer. 

7. Shadows are an important cue to depth. (See also the discussion of shading in Sec. 3.3.3 on 
The Use of Contrast.) 

8. Apparent intensity of point-light-source objects can give a cue as to range and depth. 

Thus the perception of depth can be generated by a combination of the several mechanisms described 
above. Their relative contributions to this perception are unknown, and it is therefore difficult to 
assign levels of importance to the representation of individual mechanisms in a simulator visual scene. 

Some of the effects are created without difficulty (e.g., nearly all the monocular effects listed above). 

It is difficult, however, to build hardware that permits the use of binocular vision, and most visual sys- 
tems, although they may be bi ocular, provide only monocular cues. (A notable exception is the U.S. Air 
Force's KC-135 aerial -tanker, boom-operator trainer. This device employs a physical model of an aircraft 
refueling boom viewed directly through an in-line, infinity-optics viewing system. The optical elements 
are large enough to allow viewing with both eyes. The device displays a boom image whose elements are at 
nearly the correct range from the operator's viewpoint; see Ref. 8.) Similarly, accommodation effects can 
be provided only by direct viewing of a three-dimensional model; in any system in which the image is 
"written" on a surface, whether viewed directly on a screen or through a collimation* system, all objects 
will be at the same actual range. 

However, although binocular disparity appears to be a sensitive mechanism out to quite Tong range, it 
is difficult to accept that the depth impressions resulting from binocular vision (or from accommodation 
effects) are significant to the flight task, except when near-objects are involved, such as for the tanker 
boom operator and for the final stages of landing, low-altitude helicopter operations, or close formation 
flying. Visual simulation systems providing only monocular cues certainly enjoy a level of success that 
implies that binocular depth factors are not very important for most flying tasks. 


^A binocular display is a visual simulation system that provides a separate image for each eye in such 
a manner as to create the stereoscopic cue of depth perception due to retinal disparity. A biocular display 
is a visual simulation system that provides imagery to both eyes, but the image is derived from a single 
viewpoint and, therefore, lacks depth perception cues due to retinal disparity. 
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In collimated displays, the degree of collimation has some importance in terms of viewer eyestrain 
and thus on pilot acceptance. "Comfort" limits for fusion, according to Reference 7, are of the order of 
0.5° to 1.5° of convergence, divergence, or dipvergence, divergence of the eyes (convergent light rays) 
being less tolerable than divergent or dipvergent rays. 

2.5 Mapping Factors 

2.5.1 Definitions 

The preceding section described the factors associated with the distance to objects in a displayed 
scene. This section describes the remaining dimensions. Mapping is a term used to describe where an object 
is in terms of angular measures. In any visual simulation system, location of an object can be stated in 
terms of angles to it from a convenient reference. For example, a runway boundary light may be truly 
located 20° in right azimuth and 10° negative (down) elevation from the longitudinal body axis reference of 
a landing aircraft. However, because of, say, optical imperfections, the light appears in the simulated 
scene to be displaced from the above coordinates. This effect causes the simulated scene to appear 
"stretched" or "distorted," magnified or minified, suggesting that a "remapping" has occurred. Such 
effects also introduce anomalous movements under dynamic conditions. 

The description of distortion effects is usually put in the form of vectors showing the angular dis- 
placement of a scene element from the true position. The length of the vector represents an angular devia- 
tion, and its orientation denotes the direction of the displacement. Sometimes the change is divided by 
the angle from a zero-effect reference (for example, the optical axis of a spherical mirror) and expressed 
as percent distortion, and sometimes it is divided by picture height, as is done for television monitors. 
Angular units are used in the former case and either angular or linear units in the latter. 

2.5.2 Measurement methods and illustrations 

As with the description of field-of-view boundary, two angles may be used to describe the location of 
a point in a visual scene. The order of rotation must again be specified, as must the origin of the angular 
coordinates. 

The angular location of an object in the simulated scene can be measured with a theodolite, as was 
described for the measurement of the field-of-view boundary. These measurements of scene elements can be 
compared with their true angular locations, and the errors defined as vector displacements from the true 
locations. Often a reference grid is generated and a photograph of the displayed grid is analyzed for 
the distortion factors. 

A collimated CRT display device may incorporate distortions from three major sources: geometrical 

factors, scan-line nonlinearities, and optical aberrations. The typical range of these effects is from 
zero to about 6°, or about 5% of picture height per channel. 

Unless compensation is made in the generation of the scene to be displayed on a CRT, distortions will 
arise due to the curvature of the CRT faceplate. The scene produced by a CRT must be properly distorted 
relative to its faceplate, otherwise the displayed scene appears distended, particularly at the edges of a 
highly curved faceplate. If nonlinearities exist in the scan electronics, distortion is produced that will 
add to this inherent geometrical distortion. The presence of collimating optics further distends the 
resulting image at the CRT faceplate. Figure 9 shows a photograph taken of the image formed by a CRT and 
lens collimator. The image was of an electronically generated rectangular grid with a proportion of 3:4 
between its vertical and horizontal sides. The optical axis of the collimator was coincident with the center 
of the CRT. The vector differences are Indicated. This type of collimator produces what is called "pin- 
cushion" distortion. 

Rigorous methods for determining the geometric distortion of a television camera are described in 
Reference 9. A test chart containing a regular pattern is placed before the camera. Its output signal is 
compared with that of a time-patterned signal produced by a special generator. The two are mixed (added) 
and the result sent to a monitor. The resulting picture is a graphical measure of the distortion over 
various parts of the camera field-of-view. The monitor in this case is simply a data display device and 
need not be free of distortion itself. 

A similar procedure is used for checking monitor distortions (Ref. 9). A dot or line generator is used 
as an input to the monitor. A special test mask containing pairs of concentric circles is placed over the 
monitor face. The inner circle radius is made to be 1% of the picture height, and the outer radius is 2% of 
picture height. If the dot appears within its corresponding inner or outer circle, the distortion is within 
1% or 2% of picture height for that portion of the monitor face. In this way, a graphical display of distor- 
tion is rapidly created. 

2.5.3 Importance of mapping factors 

If mapping errors are kept small, that is, less than about 6° or 5% of picture height, the Impression 
of a "skewed" world is usually not objectionable for static viewing. However, depending on the scene content, 
such levels of errors may be highly objectionable and even cause the imagery to be unusable for dynamic situa- 
tions. If the simulated world is slewing across the display field, the distortion errors may cause the image 
of the simulated world to appear "rubbery" or stretched. This can be especially objectionable in cases in 
which the scene content is small, for example, a small number of lights in a night scene. 
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Fig. 9 Collimating lens system distortion of a linear grid 


2.6 Scene Content 


2.6.1 Introduction 

The desired result of any visual simulation system is a scene, alphanumeric characters, or simple line 
drawings. Some measure of the content of scenes is needed because designers and users of visual simulation 
equipment are constantly faced with the concern for the "adequacy" of the scene content. Visual imagery is 
evaluated subjectively, and a large variety of adjectives exists reflecting points of view ranging from the 
artistic to the scientific. Objective methods cannot, perhaps, ever equal the analytical power of the human 
visual sense, but they can certainly aid it. 

In the following are descriptions of some objective measures that may serve this purpose. The measures 
are advanced without having been rigorously validated; however, it is hoped that under trial by users they 
may prove of some value. The intent of the measures described is to provide a metric for comparing the use- 
fulness of visual imagery. Although a subjective account of the surface details, using common visual adjec- 
tives, might suffice as a scene-content descriptor, it cannot be used readily in a performance evaluation 
study, whereby an objective measure might. Furthermore, it may turn out that objective measures may correlate 
sufficiently with image-generation parameters so that the "power" or "performance" or "information capacity" 
of a particular visual system could be compared with that of another. These objective measures might then be 
incorporated into a cost-performance measure. 

From the perceptual point of view, the estimation accuracy by an observer of his position and velocity 
in space might depend on the quantity of visual stimuli being viewed, as well as on their quality and geo- 
metrical array. This is supported, for example, by the preceding discussion of depth perception, where it 
was shown that quite a number of different mechanisms, relying on different features of the scene, could 
provide depth information. Thus, the establishment of imagery usefulness to the human operator might depend 
on the number of stimuli samples available. 

2.6.2 Definition 


Communication theorists have been able to quantify the content of a visual image in terms of "edge 
transitions." Carlson and Cohen (Ref. 10) have measured the spectrum of the luminance in natural scenes 
by analyzing television video signals. After assuring themselves that they were not measuring the proper- 
ties of the television signal structure, they concluded that the limi nance spectra of natural scenes could 
be closely approximated by a function that varied with the inverse square of the spatial frequency. 
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Furthermore, they observed that the power spectrum of randomly spaced luminance steps with random step 
heights also exhibits the inverse frequency- squared function. Therefore, they speculated that natural 
scenes can be thought of as an array of randomly-distributed, random height step- luminance transitions. 

This leads one to further speculate that there might be only three fundamental properties of static, 
natural visual stimuli: 

1. The number of luminance (edge) transitions 

2. The luminance difference across each edge (both magnitude and color) 

3. The sharpness of each edge, that is, the luminance gradient in a direction normal to the edge 

In the language of theoretical optics, factors (2) and (3) are combined into a metric called acutance, 
which is also related to the resolution afforded by the optical system that produced the picture. Carlson 
and Cohen developed another parameter they termed "visual capacity," a measure of scene "complexity" that 
is also related to the resolution of the complete system. It is not surprising that the fine-detail imaging 
capacity of a visual simulation system should affect its "visual capacity." Therefore, the resolution could 
be expected to set an upper limit to the potential capacity or detail count that can be produced by the sys- 
tem, or transmitted by it. 

It is recognized that the classification of visual stimuli, by measures of edge count, edge density per 
unit solid angle, contrast and edge sharpness, is an incomplete description of them; undoubtedly, their geo- 
metrical array or pattern constitutes a fourth factor. Furthermore, it is recognized that the dynamic scene 
content may be different from the static value. For example, when viewed at night, a hilltop that is covered 
with randomly located point light sources, whose ranges from the observer cannot be judged, presents a random- 
appearing array of lights to the static observer. If the observer now moves toward the hill, the number of 
luminance transitions will not change, but the relative movement of the lights permits the observer to per- 
ceive the hill shape and the approach velocity relative to the hill dimensions. If the approach velocity is 
known, the dimensions of the hill are also known. Certainly, a static measure of scene content may be an 
Incomplete description. It is speculated here that a fifth fundamental property of visual stimuli is the 
dynamic aspect of luminance transition, that is, the radial and tangential velocities of the stimuli relative 
to the two vectors in space that describe the observer's rotational and translational velocity. 

Consider, again, the nighttime scene of the hilltop covered with point light sources being approached by 
an observer. The translational movement causes the array of lights to appear as radially diverging away from 
the impending Impact point. The apparent radial velocity of each point depends on its location on the hill, 
and it maintains a unique relationship to the radial velocities of all the other points. If now the observer 
is rotated (rolled) about the line from the observer's eye point to the impact point, a tangential pattern is 
superimposed on the existing radial one. The pattern of object movement for any kind of observer movement 
will always show the vector sum of these basic patterns, and the human visual sense may be sensitive to these 
patterns as a basis for movement and form perception. 

2.6.3 Measurement methods and illustrations 

The number of luminance transitions in a given solid angle of a visual scene may be measured by an edge 
scanner that counts the observable edges in a given scan direction. Take several samples across the scene in 
two (orthogonal) directions and obtain an average number of luminance transitions in each direction. The 
square root of the product of these two numbers is a measure of the scene complexity. 

If the two scan averages are not different by more than a factor of 2, the imagery may be considered 
isotropic; that is, the statistical properties do not vary with scan direction (Ref. 10). Under these con- 
ditions, the number of isolated "patches" or "face boundaries" is approximately equal to one-half the total 
number of luminance transitions. These measurements may be used for a whole scene or for only a selected 
portion. The luminance transition density is then obtained by dividing the square root of the product, 
termed the luminance transition count, by the solid angle of the portion analyzed. 

For purposes of illustration, subjective luminance transition scans have been made of several photo- 
graphs. They range from photographs of computer-generated imagery (CGI) scenes and line drawings to those 
of real-world runways, simulated (by camera and model techniques) runways, and a wooded canyon. The angular 
dimensions of the rectangular fields of view covered were assumed to be about 48® horizontally by 36® verti- 
cally. The field size was, therefore, about 4% full field. These scenes were all scanned by eye from a 
distance such that the above angles were subtended. Illumination consisted of a 60-W desk lamp placed at about 
0.5 m from the photograph. The CGI scene was composed of about 900 edges. The ranking, in terms of luminance 
transition count and density, is shown in Table 2. All the scenes could be considered isotropic. The scenes 
themselves are shown in Figure 10. In Figure 11, the relationships among luminance transition count, density 
per square degree, and field-of-view size are shown, and the results for the photographs analyzed are plotted. 
This plot may be useful for comparing scenes subjected to scan analyses as previously described. 

The data of Table 2 are plotted in Figure 11. Note that the values of luminance transition density 
for the CGI scene are similar to those of the line drawing. Also, the values for camera/model scenes are 
higher. The difference between the values for the real runway scene and its camera/model simulation is 
primarily due to the runway texture, which in the case of the simulated scene was either absent from the 
model or not acquired by the visual simulation system. 



SCENE DESCRIPTION 

LUMINANCE TRANSI- 
TION COUNT 

AVERAGE LUMINANCE 
TRANSITION DENSITY 

Color photograph of a wooded 
canyon with stream 

190 

O.llO/deg^ 

Black-and-white photograph of a 
runway 

17 

0.010/deg* 

Black-and-white photograph of a 
camera/model simulated runway 

13 

0.008/deg* 

Color photograph of a camera/ 
model simulated runway 

10 

0.006/deg* 

Simple line drawing of a runway 
grid pattern 

7 

0.004/deg* 

Photograph of a computer-generated 
country road scene 

5 

0.003/deg* 


TABLE 2: Content factors for several photographs 



Fig. 10 Photographs of scenes analyzed: C = luminance transition 

count; D = 1 uni nance transition density; deg“^ 
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2.6.4 Importance of scene content 

FIELD SIZE 

As was stated earlier, the validity of the scene 
content metric has not been established and, there- 
fore, its importance cannot be fully assessed. It 
is not known what the variability is for different 
observers or for repeated scans by the same observer. 

Also, the variation with range, height, visibility, 
and other factors associated with natural scenes is 
not known. Speculation about these variations, how- 
ever, produces the interesting result that the number 
of luminance transitions should not vary with range 
to natural features, but should vary with illumina- 
tion and visibility. Naturally, for a perfectly uni- 
form picture (white field), the number of luminance 
transitions will be zero, and, on the other hand, an 
image composed of a checkerboard pattern will produce 
the highest number of luminance transitions if the 
checkerboard size is equal to the imaging system's 
limiting resolution. It is expected that the highest 
possible number that could be obtained for a sample 
image is the field size (in square angular units or 
steradians) divided by the square of the limiting 
resolution (in the same angular units per optical 
line pair) associated with the imaging system. 

For example, consider the conventional single- 
window display that covers a rectangular field 36® 
by 48®. The limiting resolution for such a typical 
display is about 10 arcmin per optical line pair. 

This would give a limiting number of luminance tran- 
sitions for the system of 1,728/(0.17)^ or about 
60,000. 

Other measures are possible; for example, the sum of the mean-square values for adjacent pixel** luminance 
difference or contrast (i.e., the edge luminance gradients) or any of the measures resulting from a Fourier 
analysis of the luminance variation across strips of the image. These measures will also yield numbers pro- 
portional to "scene complexity." 

Image processing and analysis is a highly developed science, practiced by those concerned with recon- 
naissance equipment for the purpose of Identifying targets, natural resources, or other features of interest. 
Most of this work is done using computer-based image analyzers with image-accessing equipment Scanners and 
digitizers, etc.). Such equipment could be used for the analysis of scene content in the visual simulation 
context. However, these tools and methods are expensive and probably not worth the investment. Certainly, 
a designer of a simulation visual system could use a complex image analysis tool to determine requirements 
for luminance and contrast based on some image samples, and, perhaps, even to establish some scene content 
measures. However, the potential for this measure is sufficiently unclear that the use of such tools is 
risky. Instead, it seems that the better approach at this time is to validate the simpler, manual methods 
first, before attempting to employ sophisticated image analysis. 

Perhaps the most appealing aspect of luminance transition count as a measure of scene content is that 

it can be obtained manually. It is a simple matter to scan by eye over a photograph or a real scene with 

the aid of a transparent straight edge and count the observable luminance changes. The degree of corre- 
spondence with the results of more automated methods is net known. 

In sunmary, the importance of objective measures of scene content lies in their potential for assisting 

in the evaluation of imagery from the viewpoints of perception or pilot performance. Also, if such measures 

correlate with generation parameters (number of processed edges, number of face boundaries, texture statistics, 
or number of edge crossings per scan-1 i ne/channel ) , the possibility is raised of defining a cost and perfor- 
mance model of a visual system. 

Furthermore, the fact that such measures may be taken of photographs of real-world scenery implies that 
a relationship might be established between real-world scenes and simulated ones that permits the prediction 
of pilot behavior in objective terms. 


PERCENT 
FULL FIELD 



Fig. 11 Number of luminance transitions, transition 
density per square degree, and field size 


'^Television systems present the displayed image by the use of a regular structured scanning of the dis- 
play surface called raster scanning. The raster is made up of a sequence of usually horizontal scan-lines, 
the number of which determines the vertical resolution capabilities of the display system. The intensity of 
the scan-line is modulated as it moves horizontally, and the maximum rate at which it can be modulated deter- 
mines the horizontal resolution capabilities of the display system. • The smallest part of the display raster 
that is controllable, which is the smallest part of a scan-line that can be modulated, is called a picture 
element or pixel . This image structure is not unique to the display, but also applies to the image generator 
and intermediate system elements. The pixel in a CGI system is a more easily identified quantity since the 
image computation is performed on a pixel-by-pixel basis. The number of scan-lines and pixels used to make 
up the image determines maximum vertical and horizontal spatial frequency or the minimum size of an object or 
target that can be represented by the visual simulation system. Linear spatial frequency refers to the number 
of cycles of a periodic pattern that can be represented across the display width. Angular spatial frequency 
refers to the number of cycles of a periodic pattern that subtend a given angle at the viewpoint of the dis- 
play; it is a more meaningful term for visual simulation. 
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3. ENERGY PROPERTIES 

3.1 Introduction 

Electromagnetic radiation covers a wavelength range of more than 20 orders of magnitude; however, the 
human eye is capable of using only a very narrow bandwidth between about 400 and 700 nm. The physical con- 
ditions on our planet have apparently caused our eyes to evolve with a sensitivity to this bandwidth. By 
virtue of its size, the eye has very low resolving power for wavelengths above 1 mm. The atmosphere rapidly 
attenuates most radiation below 1 mm except for a few wavelength windows between a few tenths of a micrometer 
and about 20 ym. The Sun emits an electromagnetic spectrum quite similar to that of a blackbody at a tempera- 
ture of about 10,400® R; the peak energy release is at about 600 nm, which is very near the middle of the 
wavelength region that we sense as visible light. It appears, therefore, that evolution has caused the human 
eye to be most sensitive to the spectral region of greatest energy reaching our planet. If the eye were 
sensitive to other regions, such as infrared, we could see thermal images in "the dark" and our definition 
of "visual region of the spectrum" would be different. 

The energy characteristics of a visual simulation system, as defined in this report, Include the charac- 
teristics of luminance, contrast, resolution, and color. Although the first three characteristics are known 
to be extremely important to the performance capabilities of pilots and utility of simulator visual systems, 
the importance of color is still debatable except from the standpoint of pilot acceptance. Luminance, con- 
trast, and resolution are very much interrelated and must be balanced carefully with respect to the tasks to 
be performed in the simulator and the capabilities of the human visual system In order to achieve optimum 
performance. These parameters are interrelated to such an extent that, in considering simulator visual sys- 
tem performance, they cannot be considered in isolation of each other. For example, (modulation transfer 
function) is used extensively as a resolution measure, even though in actuality It is a combined measure of 
spatial resolution and modulation contrast. However, the following is a short discussion of each parameter 
from the perspective of that individual characteristic. In addition, the signal -to-no1se ratio is defined 
as a parameter because of its great usefulness in highlighting performance characteristics and deficiencies 
of television-based systems. The probability of detection, recognition, or identification is a function of 
the signal-to-noise ratio of the image, the function being different for each type of task. 

Each of these characteristics is defined, and measurement techniques are suggested. In some cases, 
more than one measurement technique is suggested so that the measurements can be made to an accuracy consis- 
tent with the equipment and time available to the particular facility. The use and importance of the various 
energy characteristics are also discussed. In explaining the importance of these various characteristics, 
formal experimental evidence has been supplemented by the considered opinion of the users and operators of 
visual simulation systems. 

In view of the fact that the measurement techniques for the various energy characteristics are considered 
separately in the following sections, it is worthwhile to specify beforehand the conditions of measurement. 
Every adjustment of luminance, contrast, resolution, and signal-to-noise ratio, particularly in a television- 
based system, will change not only the characteristic being adjusted but also every other energy character- 
istic, The system can be adjusted for large values of luminance, but the resulting degradation of resolution 
and contrast will make the system unusable. Optimization of other characteristics will result in other 
similar system degradations. Therefore, it is important that all energy measurements be made at one system 
adjustment, and that this adjustment not be changed between the measurements of the various energy character- 
istics. Some systems have different optimum adjustments for different simulated conditions, for example, for 
day scenes and for night scenes. Therefore, it may be worthwhile to make a set of energy-characteristic 
measurements at each of these conditions with the system reoptimized between sets of measurements. 

3.2 Luminance 
3.2.1 Definition 

Light is a psychophysical concept based on the visual sensations arising from_ the stimulation of the 
eye by radiant energy in a certain portion of the electromagnetic spectrum. It is not therefore a direct 
physical quantity and can only be measured by reference to the response of the eye. The various parameters 
that are used to measure light are termed photometric quantities. These quantities used to be based on the 
light emanating from various types of standard candles, but are now defined by the amount of light emanating 
from a blackbody surface at the temperature of melting platinum (2042°K). The three basic quantities are 
called luminance, illuminance (or illumination) and luminous flux. The relationships between these three 
quantities can best be understood by referring to Figure 12 and to the following extract from Jenkins and 
White (Ref. 11). 
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The amount of light flowing out from a point source within the solid angle subtended 
by the element of area dA at the distance r (Fig. 12(a)) is proportional to the solid 
angle. This is found by dividing the area of dA projected normal to the rays of r^, 
so that the luminous flux in this elementary pencil may be written: 

dF = const X 

Since the source in practice is never a mathematical point, we must consider all pencils 
emitted from an element of area dS, as shown for three of these pencils in part (b) of 
Figure 12. Assuming that the source is a so-called “Lamberts Law Radiator," the flux 
will now be proportional to the projected area of dS as well, so that: 

dF = const X dS..dA cose cos» 
r^ 

The value of the constant depends only on the light source and is called its photometric 
brightness B (or luminance ) . . .. The unit of B is experimentally defined as 1/60 
of the brightness of a blackbody at the temperature of melting platinum and is called 
the candle per square centimeter,® Expressing B in this unit, the flux becomes: 

dF = B dS dA cose c . os^ , 

r2 

. . , The illuminance E of a surface is defined as the luminous flux incident per 
unit area, so that: 


.jr _ dF _ B cose dS C 0 S(J) 

‘‘E - -as p ^ 

Illuminance is often expressed in lumens per square meter, or lux. In order to calculate 
the illuminance at any point due to a source having a finite area, we must integrate 
over this area: 


E _ J J g dS cose cos(t> 

The exact evaluation of this integral is, in general, difficult but, in most cases, the 
source is sufficiently far from the illuminated surface that we may regard both cos^ 
and r as constant. In this case, 

E = ^ J J B cose dS 

_ , COS^ 

~ ^ 2 
r 

where the integral has been designated by I, since it represents what is called the 
luminous intensity of the source. 


The above extract presents one of the clearest definitions of luminance, luminous flux, and illuminance. 
It also introduces the concept of luminous intensity, which is equivalent to "candle power" as used by the 
illumination industry, and which is often confused with luminance. 

The relationship between photometric and radiometric quantities also gives rise to a considerable 
amount of confusion. As stated earlier, photometric quantities are based on the stimulation of the eye by 
that certain portion of the electromagnetic spectrum termed light. The relative response of the average 
human eye to this part of the system is shown in Figure 13. 

The two curves are called the luminosity curves for scotopic and photopic vision.® Photometric power, 
that is, luminous flux, is obtained by multiplying the radiometric power by either the photopic or the 
scotopic response function of the eye and integrating over the relevant part of the electromagnetic spectrum. 
Sources of light having equivalent radiometric power, but different spectral distributions, will, in general, 
emit differing quantities of luminous flux. 


®This definition has been superseded by the Commission Internationale d’Eclairage (CIE) (see Sec. 3.2.2). 

® Photopic pertains to vision at a sufficiently high luminance that cone receptors are involved. 

Scotopic pertains to vision at relatively low luminance so that only rod receptors are involved. 
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Fig, 13 The scotopic (V*) and photopic (V) curves of relative spectral luminosity 

The mathematical expressions relating photometric luminance to spectral radiance are given by the 
following expressions (Ref. 12): 

Photopic; 

^v "" ’^m /jgjj V(x)dx 

where : 

Ly = photopic luminance, cd/m^ (2® field) 

Le,;^ = spectral radiance in W/m^/sr/nm 

V(a) = spectral luminosity coefficient for photopic vision (2® field) 

Kjp = maximum spectral luminous efficiency (683 Im/W) 

X - wavelength, nm 

Scotopic: 

•■v = "s; 

where : 

Ly = scotopic luminance, cd/m^ 

Le, = spectral radiance in W/mVsr/nm 

V*.(a) = spectral luminosity coefficient for scopotic vision 

1C = maximum spectral luminous efficiency (1,669 Im/W) 


Unfortunately, several problems arise when these expressions are used to measure luminance. The first 
stems from errors in the original determination of V(a). Judd published a correction in 1951 (Ref. 13) 
which should be used when measuring the luminance of monochromatic lights from 400 to 450 nm or if the source 
has a significant amount of power in this region. Fortunately, few display systems used in visual simulation 
systems emit much light in this portion of the spectrum. 

The second problem concerns the size of the measuring field. The V(x) function applies to a 2® field 
of view of the eye. Frequently one wishes to measure the luminance of much larger fields. The luminous 
efficiency functions for large fields show the eye to be more sensitive to short wavelengths than the V(x) 
curve. The CIE (Ref. 12) has provided a provisional 10® curve (Yj,,(a)) which should be used for any measure* 
ments where extra-foveal fields are important. 
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The third problem concerns the level of Illumination appropriate for photopic or scopotic vision. 
Considerable errors can be Introduced if, as is usually the case, the photopic curve is used for measuring 
luminance at low light levels. The lower level for photopic measurements can only be given approximately 
since many conditions affect it. The CIE recommends that the lower limit be at least several candelas per 
square meter. Unfortunately, the scotopic curve is only accurate below about 10“^ cd/m^. The region below 
these two values is known as the mesopic region. Families of curves are available to cover the mesopic 
region, but the CIE reconmends the following approach until a satisfactory mathematical model covering all 
levels of illumination is available. 

Measure the scotopic and 10® photopic lumi nancies Ly and Ly(lO) with two detectors corrected to 
V'(x) and Yio(x), respectively, and combine them by means of the following formula to obtain the effective 
luminance L: 

, M Li + Lv(lO)" 

'■ ' M + LvUO) 

L, Ly, and ly(lO) are in cd/m^, and M has the value of 6 x lO"^ cd/m^. Ly(lO) may be estimated to a 
good approximation by Ly(lO) = 0.96 Ly + 0.04 Ly where Ly is the CIE photopic luminance for 2° field. 

The use of such a procedure has some significance when measurements are made on the night/dusk visual 
systems in which penetration phosphor CRTs are used. Luminances of surfaces may be reduced by as much as 
40X to 70 % from measurements made using only the photopic curve. 

The fourth problem arises from the fact that luminances having different colors are not necessarily 
additive. If, for example, one matched for brightness a green light to a given white light and then matched 
for brightness a red light to the same white light and finally mixed these two quantities of red and green 
lights together, it would be found that the resulting yellow light would not match twice the brightness of 
the original white light. If, however, the red and green lights are adjusted to obtain minimum flicker with 
an alternating white light, and then mixed, one would find that the resulting yellow light requires exactly 
twice as much white light to obtain minimum flicker. 

This fact represents a breakdown of Abney's additivity law (Ref. 14) and is not particularly well 
understood. Generally, however, neutral (whitish) sources will appear equally bright when their luminances 
are the same, and little difference will be observed in unsaturated (washed out) colors of equal luminance. 
Highly saturated (pure) colors relatively far apart in wavelength and having the same luminance will, how- 
ever, rarely appear equally bright. 

For a better understanding of these problems and several others, it is recommended that Ref. 12 be 
consulted. 

The foregoing discussion may seem excessively complex for the definition of a quantity as simple as 
luminance. The next section, dealing with the measurement of luminance, will attempt to clarify the various 
units in general use. 

3.2.2 Measurement of luminance 

From the definition of luminous flux given in the previous sections, luminance can be seen to have the 
dimensions of lumens per unit solid angle per unit area. The standard unit for luminance in the system is 
the lumen per steradian per square meter, sometimes called a nit. 

The CIE defined the basic unit of light as being luminous intensity and gave it the name candela (cd). 

The definition of the candela is as follows: "The candela is the luminous intensity in the perpendicular 

direction of a surface of 1/600,000 square metre of a blackbody at the temperature of freezing platinum under 
a pressure of 101,323 newtons per square meter." This unit has the dimensions of lumens per steradian allow- 
ing luminance to be measured in candelas per unit area. It should be noted that the concept of luminous 
intensity is only accurate for point sources (see previous section), and care should be taken when using this 
quantity. Several units are used for measuring luminance; the most commonly used ones are: 

1 cd/m^ = 1 nit = IT apostilbs. 

1 cd/cm^ = 1 stilb (sb) = ir lamberts(L) 

1 cd/ft^ = Ti foot-lamberts (ft-L) 

Instruments are normally calibrated in foot-lamberts or nits (1 ft-L = 3.426 nits). See Appendix B for a 
listing of photometric conversion factors. 

Instruments for measuring luminance are called photometers. Several types are available ranging from 
relatively inexpensive pocket-size devices to fairly large and expensive laboratory instruments. The smaller 
ones are quite adequate for measuring daylight visual systems and can even be used for measuring the luminance 
of light points by displaying a matrix of points, spaced one resolution element apart, of such size that it 
exceeds the measuring field of the instrument. The advantage of the more expensive systems is their greater 
versatility, which enables precise measurements to be made regarding color, luminance of very small light 
points, MTF (see Sec. 3.4), and the dynamic properties of the display. This section is only concerned with 
luminance; however, the possibility of using one instrument for the measurement of several characteristics 
is obviously a desirable goal. 

Maximum luminance should be measured with the display set to a so-called "white field." This is achieved 
by adjusting the color controls to obtain the best white, as viewed from the pilot's eye point. Some display 
input devices have an automatic setting for "white field"; however, this may not give the best white from the 
normal viewpoint because of the spectral transmission of the viewing optics. The brightness control should 
be adjusted to the setting that gives the best overall picture when viewing a normal scene. This tends to 
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be subjective, however, because luminance affects several other parameters, such as resolution (Sec. 3.4), 
signal -to-nolse ratio (Sec. 3.6), and flicker (Sec. 4.2). The shading controls should also be set for best 
overall picture. In single channel systems, variation in luminance between the center and edge of the dis- 
play of as much as 50% may be tolerated. In multiple-channel systems with continuous displays, the luminance 
variation across the boundaries would normally be set to less than 20%. 

The actual luminance measurement should be made at several points across the display and should Include 
the center, left and right edges, and top and bottom. The exact location of each point should be specified 
with the measurement, for example, 5% of picture width from left edge. Inexpensive photometers are quite 
adequate for systems operating above 0.03 cd/m^. If the visual system is to be used at very low light levels, 
that is, from 10"^ to 10"® cd/m^, a more expensive Instrument Is required. 

The approximate luminance of point light sources can be obtained by measuring the luminance of a matrix 
of light points that are separated by a distance equal to the nominal diameter of a light point. The matrix 
must cover the measuring field of the photometer, and measurements should be taken at the center and edges of 
the display as described above. An accurate measurement of the luminance of a single light point can only be 
made using a photometer that has a measuring field smaller than the diameter of the light point. Some photom- 
eters can be fitted with a Cassegrainian telescope objective, giving measuring fields as small as 10 arcsec. 

One further measurement, which should be made, is the luminance of a moving light. (In television-based 
systems, this may be different from its static luminance.) Some display systems exhibit luminance decay times 
that are longer than the refresh rate of the display. This results in an integration effect giving a greater 
luminance for static light points than for moving light points. If the light point moves a distance greater 
than its diameter during each refresh period, no integration will take place. This will normally occur at 
angular velocities of a few degrees per second. The effect is very noticeable on systems in which a TV camera 
and model -board are used to generate the image, because of the integration time of the pick-up tube. Tracking 
a moving spot is quite difficult; however, the desired result can be obtained by using a comparison technique. 

A light point of known static luminance is made to move back and forth across the center of the display at a 
steady angular rate. A second light point, whose luminance can be varied, is set at the center of the display 

such that the moving light point passes very close to it, but does not overlap. The luminance of the fixed 

light point is varied to match the luminance of the moving light point and a plot of luminance against angular 

rate can be made, if the measurement is made at several angular rates. 

3.2.3 Importance of luminance 

In the preceding sections an attempt has been made to define luminance and to suggest how it should be 
measured in somewhat rigorous terms. The following paragraphs present a discussion of the general effect of 
luminance on visual performance. 

As will be shown in Sec. 3.4, visual acuity is a function of luminance ranging from about 0.05 at 
10" ** cd/m^ to about 2.0 at 1,000 cd/m^. It will also be shown that the MTFA of a visual system is a reliable 
measure for predicting observer performance. The MTFA is the area between the MTF curve and the threshold 
detectability curve (Fig. 35). It is quite clear that luminance has little effect on the MTFA of systems 
that are limited to displaying spatial frequencies of 0.2 line pairs per arcminute and that operate at photopic 
levels, that is, above 3 cd/m^. Even at 0.3 cd/m^, the MTF of a typical system will have a considerably 
greater effect on MTFA than will the threshold detectability curve. However, if the systems can display 
spatial frequencies of 0.5 line pairs per arcminute, higher luminance will have a significant effect on MTFA. 
Blackwell's data (Ref. 15) is in general agreement with the above if, using Johnson's criteria (Ref. 16), one 
assumes that Blackwell's disk diameter is equivalent to one line pair. His data show, for instance, that, 
for changes in background luminance of from 0.3 to 340 cd/m^, the contrast threshold to detect a 5-arcmin 
disc changes from 0.07 to 0.01, whereas, for a 2-arcmin disc, the threshold changes from 0.6 to 0.06. 

Boynton and Boss (Ref. 17) performed an experiment in which the observer was asked to find a small black 
square in an array of small black circles. They found that the percentage of targets found in a given time 
varied considerably with background luminance (using 100% contrast targets), as shown in Figure 14. It can 
be seen that if an observer is expected to find 75% of the targets in less than 3 sec, a background luminance- - 
of 79 cd/m^ is required. The squares subtended 9 arcmin and the circles subtended 10 arcmin; so, if the above 
task is classed as a recognition task that requires four line pairs across the minimum dimension (Johnson 

criteria), the perception of spatial frequencies up to 0.44 line pairs per arcmin is necessary. When a 

similar experiment was repeated using a 30-arcmin target, very little difference in performance occurred 
from 1,370 cd/m^ to 0.69 cd/m^. This size target would require spatial frequencies of 0.13 line pairs per 
arcminute, using the Johnson criteria for recognition. 

Graham (Ref. 18) found that the threshold for detecting differential angular velocity decreased by a 
factor of 2.5 over the luminance range of 0.3 cd/m^ to 30 cd/m^, with very little change for any further 
increase in luminance. Salvatore (Ref. 19) found that the threshold for detecting absolute angular velocity, 
using peripheral vision, that is, at a retinal eccentricity of greater than 30®, only increased by a factor 
of about 3 over a luminance change from 23 cd/m^ to 17 x 10“^ cd/m^. 

During the integration of a simulator having a model-board visual, it was found that most people pre- 

ferred the brightness set to about 20 cd/m^, even though it was possible to achieve 100 cd/m^. No other 
visual parameter was varied, for a neutral density filter performed the attenuation. The flicker of. a 50-Hz 
display at 100 cd/m^ is quite noticeable, although European observers used to 50-Hz television would probably 
not find it objectionable. The reason for the preference for the lower brightness was probably due partly to 
the decrease in flicker, but also to the apparent increase in signal-to-noise ratio as a result of the longer 
integration time of the eye at the lower light level. Another interesting observation with this visual was 
that few people could be sure whether the display was operating at 20 cd/m^ or 100 cd/m^ unless shown the two 
modes in operation. 

This example may show an inherent. desire by the brain to match the eye resolution to the resolution of 
the scene. The low resolution of the visual will be less apparent at the lower luminance level. A similar 
observation was made by Hufnagel when he Was performing an experiment in which observers were required to 
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rank photographs subjectively having various MTFs and 
graininess (Ref. 20). It was found that observers 
adjusted their viewing distance such that the peak in 
the eye MTF matched the decreasing MTF of the photo- 
graphs. The hypothesis was advanced that the viewer 
attempts to maintain the combined MTF near unity for 
as high a spatial frequency as possible. 

These examples seem to show that observer per- 
formance will not be significantly affected by lumi- 
nance in the range of 0.3 to 340 cd/m^, providing the 
visual simulation system is limited to displaying 
spatial frequencies of less than 0.2 line pairs per 
arcminute. However, systems displaying frequencies 
of 0.5 line pairs per arcminute will require higher 
luminance than the 20-30 cd/m^ currently available. 

It is likely that, as high- resolution systems and 
flicker-free displays become available, higher lumi- 
nance will also be more desirable. 

The psychological importance of high luminance 
is in maintaining the illusion that one is watching 
a day scene rather than a dusk scene. People in the 
film industry have long been aware that a screen 
luminance of about 20 cd/m^ is quite adequate for 
depicting a bright sunny day and this seems to agree 
with simulator experience. Below 10 cd/m^, the illu- 
sion is difficult to maintain and even at 20-30 cd/m^ 
the image content, contrast, and color must be appro- 
priate for a bright sunny day. 

The Importance of very low light-level training has not been emphasized in simulators by the military 
community in the past, but it will undoubtedly become more important in the future. The important character- 
istics of the eye for very low light-level operations (i.e., 10’^ to 10"® cd/m^) are (1) no color perception, 
(2) much lower resolution required (6 arcmin per line pair), and (3) no foveal response, requiring pilots to 
look about 4® off-axis for maximum discrimination. Two other characteristics, which probably have some sig- 
nificance in training applications, are the longer integration time of the eye at low light levels and the 
rapid loss of dark adaptation when momentarily subjected to high levels of illumination. The lower visual 
requirements for low light-level simulation make this type of visual system attractive from a cost point of 
view, particularly if, as seems likely, the training can be transferred to daylight conditions. 

Luminance has a particular significance in the 
simulation of light-points. Kraft and Shaffer 
(Ref. 21) showed how the retinal image size of a 
runway light remained constant at about 1.25 arcmin 
for ranges greater than about 2,000 ft. The graph 
in Figure 15 is adapted from his paper. The apparent- 
luminance of light-points in the real world decreases 
at long ranges due to this effect. Most visual simu- 
lation systems cannot generate light-points smaller 
than 3 arcmin. Kraft found that pilots tend to fly 
lower than intended on approaches when the luminance 
of these light-points is maintained at a constant 
level for ranges greater than that corresponding to 
the 3 arcmin real-world size. When the luminance of 
the lights was attenuated with range, pilots were 
able to maintain a more correct glide slope. In 
general, it appears that luminance attenuation could 
be used to simulate increased range whenever an 
object reaches the resolution limit of the display. 

3.3 Contrast 

3.3.1 Definition 

The word "contrast" is frequently used in describing the quality of a visual system. Unfortunately, at 
least three distinct meanings can be associated with the nonnal use of the word, each of which is described 
below. 

Manufacturers of television displays have traditionally used contrast to describe the ratio of a full 
white field (Bi) to a completely dark field (B 2 ). This is an important parameter for any display, and, in 
general, a larger ratio will enable a higher quality picture to be displayed. However, the above definition 
of contrast has very little meaning with certain types of display, for either they cannot be operated at 
maximum brightness over the full picture area or the normal picture black level does not correspond to the 
minimum screen luminance. The maximum luminance for projection CRTs, for instance, is often defined over a 
relatively small area for a relatively low duty cycle. Military users in the United States have tried to 
overcome this problem by specifying contrast ratio for checkerboard patterns. This is only a partial solu- 
tion, for the brightness levels used in the checkerboard patterns still may not correspond to the normal 
operating values; they may have been obtained at the expense of other desirable parameters, such as resolution 
and color. Although contrast as defined in this manner is an important parameter for any visual system, the 
exact method of measurement should be specified, and one should be aware of any compromises that have to be 
made to obtain the stated contrast ratio (B 1 /B 2 ). 



DISTANCE FROM PILOT'S EYE, ft 


Fig. 15 Representation of light sizes in night scene 



Fig. 14 Average target acquisition curves for 
10 subjects for 100% contrast at the 
luminance shown 
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In a second definition, contrast is used to describe the difference between the brightness of a scene 
element and its immediate background. There is a certain degree of difficulty, however, in arriving at an 
exact mathematical definition of contrast. Workers in the field of visual perception define contrast as 
(Bi - B 2 )/B 2 where Bi and B 2 are the brightness of a scene element and the background, respectively. This 
formula gives values approaching infinity for scene elements much brighter than the background, and some 
workers use whichever is the greater of Bi or B 2 in the denominator. This technique has the obvious dis- 
advantage that two sets of equations are required. In television, contrast is often defined by (Bi - B 2 )/Bi, 
where Bi is the brightness of the highlights in the test pattern. For standard test bar patterns and 
cluttered scenes, Schnitzler (Ref. 22) recommends that contrast be defined as 0.5(Bi - B 2 )/Bav where B^y 
is the average luminance of the visual field. In the case of a bar pattern, Bgy - 0.5(Bi + Bz)f and the 
above expression becomes the definition of the contrast modulation C„, where 

_ B, - B2 
^ ■ Bi + B2 

These various mathematical definitions of contrast are plotted against contrast ratio in Figure 16 to illus- 
trate the differences. The expression for modulation contrast, C|j,, is used in a measurement (indicated 
schematically in Fig. 17) of the MTF which will be discussed in Section 3,4. 



Fig. 16 Differences among cotnnonly used contrast Fig. 17 Displayed modulation of typical bar pattern 
functions; Bi = target luminance, 

82 == background luminance 


The third use of the word contrast tends to be subjective rather than objective and is used to describe 
the general appearance of a picture. A high-contrast picture is one in which a high percentage of the scene 
elements are very bright and a similar percentage are very dark; a low-contrast picture is composed of scene 
elements having either similar brightness levels or uniform distribution of brightness levels. A high-contrast 
picture is not necessarily better than a low-contrast picture and may actually contain less detail. However, 
many observers prefer a high-contrast picture. This will be discussed further in Section 3.3.3. 

No mention of color has been made in the above definition of contrast, and all measurements regarding 
contrast should be made with a normal white color. The use of the term color contrast will be discussed in 
Section 3.5. 

3.3.2 Measurement of contrast 

Only the first two definitions of contrast lend themselves to direct measurement. The first is display 
contrast or large area contrast, B 1 /B 2 ; equipment manufacturers usually measure it at the center of the dis- 
play. A more useful measurement would be the average contrast obtained by taking measurements at several 
points throughout the display. This can conveniently be performed by using a checkerboard pattern consisting 
of at least 20 rectangles arranged as in Figure 18. 

The luminance of the center of each rectangle is measured and recorded, and the pattern is then reversed, 
and the measur^nt repeated. The ratio of the white-to-black readings obtained at each point is calculated 
and averaged to give the contrast for the particular display. 
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The actual measurement of the luminance levels 
should be made with a spot photometer. The choice of 
photometer will depend on the accuracy required and 
on the light levels of the display. Some of the less 
expensive instruments will give results accurate to 
within ±10%, if the black level luminances are above 
0.03 cd/m^. One further restriction is that the 
luminance level used during the measurement should 
be those judged to give the best overall picture 
during normal operations and at which luminance and 
resolution are measured. 

The second definition of contrast, that is, 
contrast modulation, is normally associated with 
relatively small objects that subtend angles of less 
than 1® at the eye. The objects may take the form 
of natural or man-made objects within the scene or 
they may form part of a test chart. The measurement 
of contrast, using test charts, is almost exclusively 
associated with the MTF of the visual system or of 
its component parts; it is dealt with in Section 3.4. 
The technique of measurement will be very similar 
but the importance of MTF is sufficient to merit a 
separate section. 
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Fig. 18 Checkerboard pattern for measuring display 
contrast 


The photometer used must have a measuring field smaller than the size of the object whose contrast is 
to be measured. The more expen:ive instruments have variable field apertures from 3° to 2', which is ade- 
quate for the visual systems currently available. If the display uses a raster structure, the aperture must 
be sufficiently large to cover several lines to ensure that the raster structure does not affect the luminance 
measurement. Contrast modulation is a system characteristic; therefore, when making a measurement all ele- 
ments of the system should be in normal adjustment. If a model -board system is being measured, the object on 
which contrast is measured should be in its normal place on the model and should be viewed under normal light- 
ing conditions. The optical probe, the TV camera, and the display should be set up according to the standard 
operating procedures. If a CGI system is being used, the object should be in the regular data base and sub- 
jected to the same data processing and mathematical algorithms as when in normal use. 


A typical luminance distribution along a line through the center of the object is shown in Figure 19. 


If possible, the background luminance should be 
measured at several points around the object to 
obtain an average reading for 82. The object 
brightness Bi should be taken as the maximum value 
for objects brighter than the background or the mini- 
mum value for objects darker than the background. 

The contrast value obtained by the above method 
should be called the static contrast ratio to differ- 
entiate it from the dynamic contrast obtained when 
the scene is moving across the field of view. In 
general, the dynamic contrast will be a function of 
the object's angular velocity relative to the nominal 
eye point of the pilot. It is obviously of greater 
interest than the static contrast measurement, for 
the scene will, in general, be moving. Unfortu- 
nately, the measurement of dynamic contrast presents 
considerable difficulty and no satisfactory technique 
has been derived for it. The ratio of the dynamic to 
static brightness of a point light source, as 
described in the section dealing with luminance, 
can be used as a multiplying factor. A more reliable 
factor would probably be the ratio of dynamic MTF to 
static MTF at the spatial frequency corresponding to 
the size of the object (if this can be measured 
reliably). 

Figure 20 shows the general effect of movement 
on the luminance and contrast of a small object. 

The effect is caused by the various integration 
and decay processes in the particular display or 
image pickup jtev ice used by the system. These pro- 
cesses are not particularly pronounced in CGI sys- 
tems using a CRT display but can be quite large in 
model-board systems using a laggy pick-up tube or 
in CGI systems using a laggy display. As can be seen 
from Figure 20, the reduction in contrast is accom- 
panied by an increase in size and a slight shift of 
the center of the object from its true position. 

These effects may have some significance in weapon 
aiming applications. 



THROUGH CENTER OF OBJECT ► 


Fig. 19 Typical luminance distribution of target 
and background 



THROUGH CENTER OF OBJECT ► 

Fig. 20 Luminance distribution of moving object' 
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3.3.3 Use of contrast 

The intelligent use of contrast in a visual simulation system conveys a considerable amount of informa- 
tion to the pilot regarding the spatial relationships among objects within the scene. The way in which this 
information is used by the brain is not particularly well understood. However, a certain amount of experi- 
mental evidence is available concerning the use of contrast to provide visual cues for depth perception, 
target acquisition, and environmental conditions. 

Contrast influences the ability of the eye to resolve detail, with resolving power being directly pro- 
portional to contrast level. In other words, the higher the contrast between a target and its background, 
the easier it is to detect a given size target. Depending on the luminance level, the eye is sensitive to 
relatively large (of the order of 10 arcmin per half cycle) sinusoidal grating targets with modulation con- 
trasts as low as 0.003. In this case, the target would have to be about 0.6% brighter than the background to 
be detected. Under the same level of luminance, the eye can detect a 1 -arcmin target grating, if the target 
is about 6% brighter than its background. 

Consequently, contrast is an important parameter in simulator visual systems designed for target acquisi- 
tion tasks. The detection and recognition of stationary objects is a complex subject to analyze and is 
affected by many factors. Biberman (Ref. 23) quotes extensively from a paper by Self (Ref. 24) dealing with 
this subject. Self's remarks on contrast are as follows: 

1. An object (or image) is not visible unless some minimum contrast is present. 

This minimum depends on both image and observer characteristics as well as on how 
zero visibility is determined. 

2. Within limits, high contrast of a target image facilitates identification. 

However, if target contrast is known in advance, low-contrast targets can be found 
more quickly in a complex background than can target objects with an intermediate 
amount of contrast. 

3. Only when both targets and background are without internal details or con- 
trasts (i.e., when they are each of a different but uniform lightness or brightness) 
can a single ratio or contrast number be a unique and exhaustive description of the 
contrast of the target with its background. 

4. When the percentage of targets detected or the portion of responses that 
are correct are used as performance measures, contrast variations appear to have 
little effect, as long as very low contrasts are not involved. 

High-contrast situations are usually not encountered in the real world foh normal flight tasks except 
for night situations with airport lighting, and with brightly painted cultural objects. Air-to-air combat 
targets at normal detection ranges are usually not high-contrast targets because of the paint schemes used 
and atmospheric attenuation. Air-to-ground targets are, by design, usually low-contrast with respect to 
their background. Data on the performance of the eye indicate that it may be possible to enhance detection 
and recognition ranges in a simulator visual system by providing higher than real-world target contrast while 
using luminance levels below real-world levels. This could be an important tradeoff to consider for some 
future air-combat visual simulation systems, since high luminance levels (above 30 cd/m^) are difficult to 
achieve. However, it appears that the use of contrast to control the degree of difficulty of target acquisi- 
tion in complex backgrounds may have unpredictable results. 

Although in general it may seem that it is desirable to strive for the highest possible contrast perfor- 
mance, high-contrast scenes in some visual simulation systems can cause problems that result in quantizing 
effects being greatly exaggerated. Such effects are very apparent in computer-generated scenes where fairly 
bright narrow faces, such as runway markings, or texturing are used. The typical solution, although not 
ideal, is to reduce the luminance contrast. 

It was shown in Section 3.2 that scene luminances of 20-30 cd/m^ are quite adequate for depicting bright 
sunny environments provided they are colorful and have the correct range of contrasts for a bright sunny day; 
that is, walls facing the Sun must be several shades brighter than the adjacent walls, and strong shadows 
should appear under trees, etc. The gradual diminution of contrast as a function of distance from the view- 
point gives rise to aerial perspective, a well-known depth cue. Shadows and shading give strong depth or 
spatial cues, as can be seen from Figures 21-23. The drawing in Figure 21(a) can be perceived either as a 
hole in a wall or a protuberance from a wall. The use of contrast in Figures 21(b) and 21(c) to simulate 
the shadows and surface brightnesses caused by normal illumination from overhead reduces the ambiguity. The 
brain seems to expect illumination to originate from overhead sources such as the Sun and tries to Interpret 
scenes accordingly. The same effect is seen with the buttercup in Figure 22, which appears convex because 
the picture is upside down. If the picture is rotated 180°, the normal concave shape will appear. Most 
people are able to see Figure 21 as a concave or convex shape even though the shadows and shading reduce the 
ambiguity, and Figure 22 may also be perceived correctly or incorrectly independent of its orientation. It 
is interesting that once they have been perceived incorrectly, it seems quite natural for them to be seen in 
this manner. 

In the real world, the ambiguity seldom appears, presumably because of the physiological reinforcing 
cues of stereopsis, convergence, and acconmodation. Figure 23 shows another use of shadows to fix the loca- 
tion of objects in space. The position of the objects without shadows is indeterminate relative to the 
ground plane, whereas those objects with shadows are either definitely on the ground or floating above the 
ground depending on the position of the shadows. 

The importance of shading and shadows is well recognized in both model -board and CGI systems and can 
make the difference between a scene being considered acceptable or unacceptable by the user. The use of 
shading and shadows to simulate both Sun angle and indirect lighting from clouds or sky is quite important. 

It is particularly so in visual simulation systems, which have no direct physiological cues and rely entirely 
on psychological cues for depth perception and spatial orientation. 



24 




F1g, 21 Use of shading and shadows to assist 
depth perception 



Fig. 22 Effect of illumination on perception 




Fig. 23 Use of shadows to locate objects relative 
to ground plane 


The ability of a visual simulation system to generate a sufficient number of shades of gray to duplicate 
the complex patterns of light encountered in the real world is evidently important. Ten shades of gray, each 
brighter than the lower shade by a factor of v'Zi are considered to be adequate. No data seem to be available 
regarding the minimum or optimum number. 

A related problem in model -board systems is the adjustment of the system gamma. The output brightness 
function of the display is related to the input brightness function of the camera by the formula 

BquT ° 

where y is called the system gamma; it should normally be unity. A gamma of greater than unity will usually 
allow more detail to be seen in the whites of the picture and a gamma less than unity will enable more detail 
to be seen in the picture blacks. 

The first definition of contrast, that is, display contrast, is the relevant parameter for determining 
the number of shades of gray that can be used in the display. Some displays, such as the GE light-valve and 
the Eidophor, have large area-contrast ratios (100:1). Projection CRTs, which have to operate at high power 
levels, may have contrast ratios as low as 10:1. The standard type of CRT will normally allow 10 shades of 
gray to be displayed, which is equivalent to a contrast ratio of 32:1. 

An important psychological effect caused by contrast changes across surfaces is known as the Mach band 
effect, discovered by Mach in 1865. If a gray surface is illuminated such that the luminance distribution 
across the surface is as shown in Figure 24, observers report the existence of a dark line at A and a white 
line at B. If the luminance gradient is made to vary smoothly, as in Figure 25, observers report seeing 
light bands at those parts of the light distribution curve that are convex to the horizontal axis and dark 
bands at the parts that are concave. This effect is not usually seen in the real world but is sometimes seen 
in CGI systems due to their technique of surface shading. 

Consequently, an important parameter in CGI systems is the total number of discrete shades of gray that 
can be generated. In a model-board system, the transition from minimum to maximum luminance is a smooth 
function because of the analogue nature of the system. The digital nature of CGI systems prevents such an 
infinitely smooth transition. Normally 200 or more discrete steps are required to prevent unwanted effects, 
such as Mach banding, and to allow smooth shading across surfaces. 
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Fig. 24 Luminance distribution causing Mach bands Fig. 25 Luminance distribution across a surface 


3.4 Resolution 
3.4.1 Definition 

Resolution is probably the most important parameter of a visual system. It is difficult, however, and 
perhaps impossible to give a unique definition of resolution. 

The most general definition that can be given is to state that the resolution of a visual system is its 
ability to present small, recognizable details. Such a definition obviously relies on the performance of a 
human observer and it is this performance that must be defined first. The ability of a human eye to recognize 
fine detail is termed visual acuity. The various types of visual acuity are described and defined where pos- 
sible in the following paragraphs. 

The minimum separable acuity is a measure of the eye's ability to separate two small objects. It is 
measured and defined in terms of the angular size of the smallest character that can be recognized. Most 
references to visual acuity refer to this particular type of acuity and it is normally referred to as V.A. 
Upper-case letters such as E are frequently used on charts for measuring V.A. and can usually be considered 
as being made up of five elements; that is, E has three horizontal bars and two spaces. Visual acuity is 
defined as the reciprocal of the angular size of each element (measured in minutes of arc) of one of these 
letters. Normal V.A. ic 1.0; that is, a letter E subtending 5 arcmin should just be recognizable by the 
observer. Acuity is frequently expressed as the ratio between the distance to the target letter (usually 
20 ft), and the distance at which the smallest letter recognizable by the observer would have to be placed 
to subtend 5 arcmin. Therefore, a V.A. of 0.5 or 20/40 would mean that the minimum recognizable letter 
subtends 10 arcmin. Such measurements are usually performed with high-contrast targets at normal levels of 
illumination. 

Figure 26 shows how the acuity of the eye is reduced by reducing the contrast of the target. Figure 27 
shows how visual acuity is dependent on object luminance. 



Fig. 26 Object contrast modulation, aB/2Bav» neces- 
sary for eye to resolve a pattern of alter- 
nating dark and bright bars of equal width 
(B^y = 35 cd/m^) 



Fig. 27 Variation in V.A. with object luminance 
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Figure 28 shows the variation in visual acuity 
with retinal position of the image at photopic lumi- 
nance levels. This variation is due to the low con- 
centration of cones in the peripheral areas of the 
retina relative to the fovea. The lower acuity in 
the peripheral field of view is of great importance 
in simulator visual systems, such as area-of-interest 
displays, that attempt to match the image resolution 
and detail to the acuity of the eye. 

The minimum perceptible acuity refers to the 
ability of the eye to detect a small object in a 
large field of view. The narrowest black line that 
an eye can detect on a bright field subtends an angle 
of 0.5 to 1 arcsec. Such performance seems to rely 
on the ability of the eye and brain combination to 
integrate along the line. For very bright spots, 
the size is not as important as the brightness. 

The limiting factor is the number of photons col- 
lected by the recipient retinal cell in a given 
time span. 

Vernier acuity is the ability of the eye to align two objects, such as two straight lines. The average 
eye can accomplish this task to within about 5 arcsec. 

The ability of the normal human observer to perceive depth in a scene due to the separation of the eyes 
is called stereoscopic acuity . An average observer can adjust the position of two rods in a clueless environ- 
ment at 6.1 m to within 2.5 cm. This corresponds to a difference between the angles of eye convergence for 
each object of about 10 arcsec. 

The lower limit for stereoscopic acuity depends on the angular separation of the rods, their length, 
their thickness, and their proximity to other objects. One parameter that has a surprisingly small effect 
over a large range is visual acuity. In 1938, Matsubayashi (Ref. 25) found that reducing the visual acuity 
in one eye to 0.3 had little effect on stereoscopic acuity. Further reduction to 0.2 caused a considerable 
decrease in stereoscopic acuity and at a V.A. of 0.1 in one eye no depth perception was possible. Researchers 
at Boeing Aerospace Company have recently obtained similar results when the acuity of both eyes was reduced. 

One further type of acuity is the ability of the eye to detect motion. The slowest motion that can be 

detected is about 1-2 arcmin per sec. The more general case of this type of acuity, that is, the ability to 

detect a change in an object's angular velocity with respect to the observer. Is described in Reference 26. 

Figure 29 shows how the differential threshold varies with the angular velocity of the object. The differ- 
ential threshold (aw) Is the amount that the angular speed of an object moving at right angles to the line 
of sight must change to be detected at a new speed. Data points shown on Figure 29 are thresholds gathered 
from eight different experiments, for abrupt changes in speed from n>i to u) 2 . 

Threshold for movement In peripheral vision is higher than the threshold in central vision. Effects 
of Illumination and contrast on differential threshold are Imperfectly known at this time. The rate threshold 
Is higher at low illumination levels and when no fixed visual reference is available. 

A related type of acuity is the ability to detect differential movement between objects in the visual 
field. This occurs when an observer is moving through space and fixates on one of two neighboring stationary 
objects. The resulting parallax between the objects gives rise to the very strong depth perception cue known 
as monocular movement parallax. Graham and others investigated this cue in 1948 and produced a graph based 
on experimental data showing the relation between the threshold for differential angular velocity, (in 

arcseconds per second), and the actual angular velocity of the objects relative to the observer. This curve 
is shown in Figure 30. A full description of the experiment is given in Reference 18. 




Fig. 29 Threshold for differential angular velocity as Fig. 30 Threshold values wt as a function of the 
a function of the objects' angular velocity rate of movement of the stimulus objects 
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All of the above types of acuity are relevant to visual simulation system design; however, the discus- 
sion of the measurement of resolution will be restricted mainly to that corresponding to the first type of 
acuity, that is, minimum separable acuity. The ability of a visual simulation system to provide cues depen- 
dent on these other types of acuity is usually a function of its basic resolution, although the relationship 
is seldom linear and may depend on several other parameters (as was shown in the discussion on stereoscopic 
acuity). Temporal properties will also have a pronounced effect on the types of acuity related to motion; 
these are discussed in Section 4, It is outside the scope of this report to try to establish these relation- 
ships or even to suggest how they should be measured. It is important, however, to realize that other types 
of resolution do exist and that they probably affect pilot (or observer) performance. 

3.4.2 Measurement of resolution 


The concept of resolving power was first applied to optical devices by Lord Rayleigh (Ref, 27). He 
defined two, equally bright, point sources of light as being resolved when the center of the diffraction 
pattern of the first fell into the first dark ring of the diffraction pattern of the second. Figure 31(c) 

illustrates Rayleigh's criterion, which is the most 



(a) NOT RESOLVED (b) BARELY RESOLVED 


X - WAVELENGTH OF LIGHT 
d - DIAMETER OF APERTURE 




(c) RESOLVED id) CLEARLY RESOLVED 

Fig. 31 Resolving power. (The dashed lines repre- 
sent the diffraction patterns of two point 
images at various separations. The solid 
line indicates the combined diffraction 
pattern.) ' 


widely used for defining limiting resolution of 
optical systems. Figure 31(b) corresponds to Sparrow's 
criterion (Ref. 28), which can be used under ideal 
viewing conditions. 

The concept of limiting resolution has been used 
by the television industry for many years. It is nor- 
mally measured by displaying a wedge pattern of alter- 
nate black-and-white lines and determining the line 
spacing where the patterns can just be resolved. It 
is a relatively easy measurement to perform; unfortu- 
nately, it is an unreliable measure of picture quality. 
Differences in measurement of up to 25% can be obtained 
merely by changing the length and number of lines in 
the wedge. The shrinking raster technique is also used 
to measure resolution. In this technique, a raster of 
equally spaced lines is drawn on the display and grad- 
ually "shrunk" until the raster structure disappears. 
Resolution measurements obtained by this method are 
typically 40% lower than those obtained by the limiting- 
resolution method. 

The most widely accepted method for measuring 
resolution is the MTF technique. In this method, a 
sinusoidally modulated test pattern, having a variable 
spatial frequency, is used. The input test pattern is 
100% modulated and the output modulation is plotted 
against spatial frequency, giving a plot similar to 
that shown in Figure 32. For practical reasons, the 
lowest spatial frequency used in this measurement 
normally corresponds to about 10 cycles per picture 
height. The remainder of the curve is obtained by 
extrapolation. If, as is often the case, the input 
test pattern is modulated by a square-wave function, 
the resulting curve should be called the contrast 
transfer function (CTF). 


Such a curve gives an adequate description of the resolution characteristics of most displays. It can 
be used to predict operator performance for a number of visual tasks, if the signal -to- noise ratio of the 
system is sufficiently large. 


Slocum et al. (Ref. 29) calculated the relationship between various resolution techniques for a CRT 
assuming a spot with a Gaussian brightness distribution for the electron beam. The result is shown in 
Figure 33. 


A major advantage of the MTF technique is that MTFs of cascaded devices can be multiplied together to 
give a system MTF, provided each device does not compensate for aberrations in the previous devices. MTFs 
of optical systems can be calculated, using computer programs to perform the extranely laborious calculations. 
Of interest in model-board systems is the maximum MTF of a diffraction-limited optical system. This is 
given by 


2 k 

MTF(v) = - (4 - cos 4 §in 4)(cos e) 

IT 

where 4 = cos"^ (x vf), X is the wavelength in millimeters, v is the frequency in cycles per millimeter, 
f is the relative aperture of the optical system, 0 is the half-field angle, and k = 1 for radial lines 
and 3 for tangential lines. A plot of this curve is shown in Figure 34. 

The MTF will be zero when 4 is zero, that is, when 

V = s Vq (f is the relative aperture of system - F/d) 
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SPATIAL FREQUENCY (N), half cycles per unit distance 


Fig. 32 Typical MTF curve 


Fig. 33 Relative modulation transfer function 


For large relative apertures, that is, f > 5,' the limit of the angular resolution is given by 


where F = focal length and d is the diameter of the aperture. For an optical probe having a 1-mm 
entrance pupil, A is equal to about 2 arcmin. An optical probe having an MTF equal to 70% of the diffrac- 
tion limited curve at the higher spatial frequencies would be considered to be very good. 

In general, optical systems will have a different MTF for radial and tangential lines; this is usually 
a result of the image plane being different for the radial and tangential components. As viewed directly by 
the eye, this difference may not be important because the eye can accommodate. An extreme case of this prob- 
lem is shown in Figure 35 for a pupil-forming collimated display. Further difficulties arise when measuring 
RTFs on displays that are raster processes (e.g., television displays). The MTF parallel to the raster 
lines (or at any angle other than 90° to the raster lines) will be seriously affected by the raster structure 
at spatial frequencies greater than half the raster line frequency. 


MEASUREMENTS TAKEN 
4.25" OFF APERTURE 
AT A 25" FIELD ANGLE 



Fig. 34 The maximum modulation transfer function of 
a diffraction- limited system. Note that 
the curve is based on diffraction effects 
for a circular aperture and that the 
abscissa is the ratio of the frequency to 
the limiting frequency, Vq. 



Fig. 35 Contrast modulation curve for a wi de-field 
pupi 1 -f ormi ng display 


The measured MTF will be a function of the position of the bar pattern relative to the raster. In 
effect, the raster structure interferes with the bar pattern. The effect, called aliasing, is dealt with in 
Section 4.5. This problem will be encountered in any system using sampled imagery, such as a TV camera, a 
CGI system using digital computation techniques, a raster display, or the shadow mask of a color TV CRT. 
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Although the MTF curve is an essential measurement for a visual system, it does not give a unique fig- 
ure of merit enabling comparisons to be made between visual systems. Chapman and 01 in (Ref. 30) proposed 
the use of the area between the MTF curve and the threshold detectability curve as a measure of image quality 
for photographic systems. The metric was called the modulation transfer function area (MTFA) and has been 
applied to electrooptical systems in general. The threshold detectability curve would normally be that of 
a human observer but may be modified by other system characteristics. Figure 35 shows typical MTF curves 
for an optical display system together with a threshold detectability curve for a typical observer. The 
areas between the threshold detectability curve and the MTF curves represent the radial and tangential MTFA 
for this particular system under the conditions of the measurement. The MTFA concept has been used with 
great success in measuring image quality in general and in predicting operator performance for many visual 
tasks. (See, e.g., Ref. 23, pp. 106-118.) 

The foregoing techniques for measuring resolution are normally applied to static images. Unfortunately, 
the image presented to a pilot on a visual simulation system is seldom static, and a much more relevant 
parameter would be the dynamic resolution. In general, this will be a function of image velocity across the 
display measured in terms of seconds per picture width. Since fairly high rates of target image movement 
can arise in some of the more demanding air-combat tasks to be simulated, the dynamic resolution capabilities 
of the visual simulation system can be very important. Degradation in resolution results from sources such 
as lags in camera tubes, lags in display tubes or projectors, and separation of the imagery in interlaced TV 
systems. These are discussed further in Section 4. 

The degradation of resolution as a function of image movement was shown by Heintzman in the comparison 
of a vidicon and a plumbicon camera tube under dynamic conditions (Ref. 31). Data from his work are shown 
in Figure 36. This figure shows that, for the static condition, the vidicon has a much higher MTF than the 
plumbicon. However, as the rate of movement increases to about one picture width in 12 sec, the MTF of the 
vidicon decreases rapidly and is nearly the same as the MTF of the plumbicon, which has decreased only 
slightly. As the rate of movement increases further, the MTFs of both drop off dramatically. These mea- 
surements were made with the two different tubes in test cameras observing a moving belt containing resolu- 
tion patterns. The MTF data were taken from an oscilloscope looking at the camera video output. 

The dynamic resolution of model -board systems using low- lag displays such as CRT*s is determined almost 
entirely by the pick-up tube, and a satisfactory measurement of dynamic MTF can be obtained by measuring the 
video output of the camera, as done by Heintzman. It should be noted that the display gamma and MTF (static) 
must be taken into account to obtain a valid system measurement. The resulting plots of resolution for var- 
ious scene velocities will be useful in predicting operator performance for certain visual tasks but are 
still unsatisfactory as a standard measurement for resolution. 

Two procedures for measuring MTF are given in Section 3.4.3; however, neither lends itself to measuring 
dynamic MTF, 


Another important factor affecting dynamic resolution is the temporal quantization of the visual inf or 
mat ion due to the field rate. Each field time (1/60 or 1/50 sec), a new picture is drawn on the display. 

If the scene is moving horizontally at a certain rate, then vertical bars of a given spatial frequency may 
move an amount in each field corresponding to the distance between a white and a black bar. Under such con- 
ditions, the MTF for this spatial frequency will be zero. Although an observer may still see a moving bar 
pattern, its spatial frequency and velocity will not be correct. This phenomenon causes the stroboscopic 
effects sometimes seen on television systems — for example, wheels turning backward and runway markings 
moving relative to the runway. The MTF curve for a given scene velocity will be as in Figure 37. 


STATIC *12 seconds 



*TIMES SHOWN INDICATE TIME REQUIRED FOR DYNAMIC 
IMAGE TO MOVE FROM THE RIGHT EDGE TO THE LEFT 
EDGE OF THE CAMjERA FIELD OF VIEW 

Fig. 36 Dynamic resolution 



Fig. 37 MTF curve for a moving bar pattern on a 
television display 
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Spatial quantization of the video information is another important factor affecting dynamic resolution. 
The mechanics of spatial quantization will be described under "aliasing" in Section 4.5. Several software 
techniques that are used to make the effects of quantization less objectionable have an effect on both 
static and dynamic resolution. 

As stated earlier, it is the MTFA rather than the MTF that has been found to be a successful metric 
for resolution of static imagery. The measurement of the threshold detectability curve under dynamic con- 
ditions is probably even more difficult than measuring the dynamic MTF. The eye is normally stabilized in 
space by the vestibular-ocular reflex. This reflex takes the form of a direct servo loop between the 
vestibular sensors in the ear and the eye movement muscles. It is very effective, at frequencies up to 
10 Hz, in stabilizing the aiming point of the eye under vigorous head movement. This fact can be verified 
by shaking one's head while reading this page. It will be found that the letters are still legible. If 
the page is shaken at the same rate and the head kept still, the letters will become blurred and probably 
illegible. This demonstrates the relatively poor visual acuity of the eye when tracking rapidly moving 
objects (Ref. 32). A contributing factor may be inhibitions of vision which seems to occur during saccadic 
eye movements (Refs. 33, 34). If the object moves at velocities lower than 20°/ sec, the eye can follow 
fairly smoothly, although small saccades still occur especially during periods of high acceleration 
(Ref. 35). 

In a fixed-base simulator, high angular rates are simulated by appropriate changes in the visual simu- 
lator system. It is quite evident that the performance of the eye under these conditions will be different 
from that in a real aircraft. The use of a motion system may reduce the difference under certain conditions, 
but may also aggravate the situation if, as is usually the case, only onset cues are provided. 

From the foregoing discussion, it seems apparent that a meaningful measurement of resolution should 
take into account the physiological and the psychological characteristics of the observer under dynamic con- 
ditions as well as the physical and software characteristics of the simulator visual system. A plot of MTFA 
against scene movement would accomplish this and provide useful data for both user and designer. However, a 
technique based on the recognition of characters, as used in the measurement of visual acuity of the eyes, 
would be easier to perform. Such a system would be oriented toward users rather than designers. The proce- 
dure would consist of causing letters of known size to appear on the screen for a fixed time interval and to 
move at given velocities. A number of observers having normal visual acuity would be asked to identify the 
letters; the letter size that was correctly identified 50% of the time would then give the V.A. of the system. 
A curve such as that shown in Figure 38 would prob- 
ably be obtained. This curve should enable users 
to determine which visual tasks could be performed 
successfully on a given simulator visual system. 

As stated previously, the MTF of a display 
varies considerably with both field angle and the 
orientation of the bar pattern relative to the axis 
of the display. Resolution measured by the V.A. 
technique would not be affected to the same extent 
by the angle of orientation, for a deficiency in 
resolution in one direction will be corrected to a 
certain extent by greater resolution in the orthog- 
onal direction. A single value for V.A. could be 
obtained by causing the letters to appear throughout 
the viewing field and at different angles. Kraft 
described a similar technique for evaluation of a 
CGI system at an A6ARD Conference in 1978 (Ref. 21). 

An investigation and evaluation of this technique 
by the simulation industry appears to be warranted. 

3.4.3 Procedure for measuring MTF 

Two techniques will be described, one using a photometer and the other developed by Systems Research 
Laboratories for the U.S. Air Force, using a television camera. Before performing any resolution measure- 
ments, the visual system should be adjusted for best picture. 

Certain photometers can be fitted with an accessory that allows MTF to be measured directly. The 
photometer is placed at the nominal eye position of the visual simulation system and caused to scan across 
the displayed image of a bar pattern. The output from the photometer is recorded on a strip-chart recorder. 
Measuring fields as narrow as 0.4 arcmin can be obtained so that spatial frequencies as high as 0.5 cycles 
per minute of arc can be used. A similar accessory is described in Ref. 36 for measuring the MTF of CRT's. 
This device scans laterally and can only be focused a short distance away from the photometer. The measure- 
ment of MTF from the pilot's eyepoint requires the use of a radial scanner that can be used with collimated 
images. 

Systems Research Laboratories developed another technique for the U.S. Air Force that has been used 
successfully on several visual simulation systems. A high-resolution monochrome TV camera with a long focal 
length lens is placed at the pilot's eyepoint and focused on the visual display. Bar patterns corresponding 
to various spatial frequencies are displayed and the video from the camera is connected to an oscilloscope. 
The focal length of the lens must be sufficiently long that the image of the bar pattern on the faceplate 
for the TV camera corresponds to a low spatial frequency relative to the picture width of the camera. This 
is to ensure that the MTF of the camera does not have a significant effect on the measurement. Even with 
this precaution, the MTF of the camera and lens should be known if a very accurate measurement of the visual 
system MTF is required. If the camera has a unity gamma, the peak-to-peak measurement of the waveform 
obtained on the oscilloscope at the various spatial frequencies will enable an MTF curve to be plotted 
directly. 
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Using one of the above techniques, the MTF should be measured at the center and several selected 
points on the edge of the display. In systems that exhibit depth-of-field effects, either in the taking 
or viewing optics, the measurements should be repeated at points in the display representative of the task, 
for example, at several slant ranges along the runway centerline for a landing simulator. Once a series of 
measurements has been started, the operating controls of the visual system should not be changed. However, 
the focus control of the measuring device should be optimized for each location. Whether the measuring 
device should be focused separately for the vertical and horizontal measurements is debatable and probably 
depends on the application of the system. If there are significant differences in the focal planes of 
vertical and horizontal or radial and tangential lines, two sets of curves should be obtained and the 
focusing method recorded. 

Using a standard threshold detection curve for a human observer (or standard pilot), the MTFA can be 
calculated for each MTF curve. If the MTF is reasonably constant over the field of view, the central MTFA 
(averaged for vertical and horizontal lines) together with a percentage deviation can be used as a unique 
figure of merit. 

In general, the MTFA measured from viewpoints away from the design eye point will be lower and should 
also be measured. 


3.4.4 Use of resolution 


Resolution is a self-evidently important characteristic of visual simulation systems. The basic 
requirement of any system is to furnish the simulator pilot with the visual cues necessary to carry out the 
simulated task. Once those cues have been established, they can be reduced to engineering units of angular 
subtense and the smallest angular subtense becomes the resolution requirement of the system. The importance 
of resolution is so apparent that a few words of caution are advisable. Resolution and contrast are inter- 
related. Any optical system operating near its limit of resolution will suffer a decrease in contrast; the 
sharpness of the picture will decrease. An increase in electronic system bandwidth to accomnodate an increase 
in resolution and picture information will also admit unwanted noise and the quantizing effects of CGI sys- 
tems. A careful estimation should be made of this unwanted noise and an estimation of whether its addition 
will really result in decreased picture sharpness. Artificial enhancement of contrast in the image-generation 
equipment may overcome the effect of some of the noise sources. 


CLEAR Shown in Figure 39 is a three-dimensional surface 

SEEING depicting the relationships between resolution, con- 

trast, and luminance. Although it is apparent from 
this figure that contrast has a much greater effect 
on usable resolution, luminance may become important 
in air-combat simulators where the task demands a 
usable resolution in the range of 1-2 arcmin. Striving 
to provide high spatial resolution for such a system 
without appropriate luminance levels will most likely 
result in less than the full spatial resolution capa- 
bility of the visual system being usable. The psycho- 
physical data represented in the figure indicate that 
lower than real-world luminance levels may be compen- 
sated for by providing higher than real-world contrast 
ratios in order to provide a given usable spatial 
resolution. 

With optimal lighting conditions of high bright- 
ness and high contrast of a black object on a white 
background, the eye can resolve targets slightly 
smaller than 1 arcmin. Fortunately, many flying 
tasks do not tax the resolution capability of the 
eye, and visual systems with resolutions of 5-10 arc- 
min per TV line have been used with much success for 
tasks such as takeoff and landing, aerial refueling, 
and formation flight. One such principal use of 
visual display systems in civil aircraft training 
applications is to represent takeoff, approach, and 

landing in a wide variety of meteorological conditions. Military training requirements introduce a much 
wider variety of tasks, including weapon aiming, in which the maneuverability of the aircraft is exploited. 

If full -miss ion simulation is to be accomplished, military aircraft simulators need a greater field of view 
and higher display resolution than their civil equivalents. 



Fig. 39 


This curved three-dimensional surface shows 
the relationship between visual angle, con- 
trast, and background brightness. When a 
combination of these factors falls on a 
point below the curve, we cannot discrim- 
inate; a combination falling on a point 
above permits easy seeing. 


The only solution to the problem is to compromise both the range of tasks that can be represented and 
the quality of representation of the outside world. One example of this compromise is the use of the single 
front-window display obtained from model -board/ television systems, which restrict landing simulation to 
straight-in approach. A second example is the use of the beam-penetration cathode ray tube, which gives a 
high-resolution night scene but are severely limited in their ability to create daylight scenes. The success- 
ful simulator is the one that achieves an acceptable compromise in matching the simulated task to the resolu- 
tion characteristics of the visual display device. Similarly, if the expectations of the pilot exceed the 
capability of the simulator visual system (as in the case of using model -board/television systems for ground 
attack training), then the simulator will be criticized and not utilized. 


The effort and resources being devoted to technology development in the improvement of resolution of 
simulator visual systems is indicative of the importance placed on this characteristic as a means of achiev- 
ing a successful visual simulation capability for certain high-skill air-combat tasks. Whether it is essen- 
tial that simulator visual systems be able to provide a resolution capability approaching that of the eye 
for learning, practicing, or researching certain flight tasks is still an unanswered question. If the pilot 
of a simulator is to perform an air-combat task in the simulator in the same manner that he performs it in 
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an aircraft, the argument can be made that he must be able to detect, recognize, and identify targets in 
the simulator at ranges equivalent to those in actual flight. This argument would require that the simu- 
lator visual system match the resolution capability of the eye. Thus, the question is essentially reduced 
to the degree of simulator performance-equivalence required for training, or for practicing a given task, 
or for performing research on the task or on a flight vehicle. 

The ability of a visual simulation system to be used in closed-loop control tasks is related to its 
resolution. The control of an aircraft requires a complex set of visual cues including monocular movement 
parallax, differential velocity discrimination, peripheral vision motion detection, and the changing 
angular size of objects moving toward the observer. Associated with each of these motion-related cues is 
a threshold velocity which sets the limit on the precision with which a given control task can be performed. 
The simulation of this velocity in a television-based visual system requires a discrete positional change 
for one or more objects at appropriate intervals of time. The resulting perceived motion, called apparent 
motion, is a complex subject; it is discussed in Section 4. However, if we assume the time interval must 
be 200 msecs or less, the simulation of the threshold velocity for monocular movement parallax, that is, 

30 arcsec/sec (Ref, 18), would seem to require positional increments of 6 arcsec. The threshold for detec- 
tion of motion in the peripheral vision is about 15 arcmin/sec at 30® from the fovea (Ref. 10). Using the 
200-msec time interval, positional increments of 3 arcmin seem to be required. Exactly how important these 
thresholds are in research or training simulators is not known but they should be considered. 

3.5 Color 

3.5.1 Definition 

Color can be regarded either as a psychophysical concept (similar to the manner in which light itself 
was defined in the section on luminance) or as a characteristic of radiation (within the visual spectrum) 
and of objects. The normal human eye is stimulated by electromagnetic radiation 1n the wavelength range of 
400-700 nm. The visual sensation produced by this portion of the spectrum varies according to the wavelength, 
giving rise to the color sensation of violet through blue, green, yellow, and red. A mixture of light across 
the total spectrum will cause a sensation of no color, that is, what is called white light. Objects that 
reflect or transmit light as a function of wavelength are said to be. colored and are given the name of the 
dominant color in the reflected or transmitted light when they are illuminated with a white light. 

The current physiological theory of color vision is based on the premise that three different types of 
cones exist in the retina, each having a maximum spectral response corresponding to red, green, and blue. 

This theory is based on the work of Young (Ref. 37) and on the later work of Helmholtz (Ref. 38), which was 
carried out during the previous century and has been supported by many other researchers during the present 
century. Experiments on the absorption spectra of individual cones by Marks and MacNichol in 1964 (Ref. 39) 
and by Brown and Wald in 1964 (Ref. 40), support this theory. The former researchers found that cones in 
monkeys and humans fall into three categories having peak absorptions at 445, 535, and 570 nm. Brown and 
Wald found single cones with peak absorptions at 450, 525, and 555 nm. 

Despite the general acceptance of this theory, it fails to predict certain experimental results, and 
a considerable amount of work is being performed to find a more accurate model for color vision. (See, e.g.. 
Refs, 12, 18, and 41.) It is unlikely that any developments regarding color vision in the near future will 
affect the design of simulator visual systems. It is probable, however, that a better understanding of 
color vision will affect the way in which simulator visual systems are used. 

Color can be specified in terms of its dominant wavelength, purity, and brightness. The dominant wave- 
length is the wavelength of the pure spectral source that has the same hue as the sample. Purity is a measure 
of the amount of white light that has to be mixed with the pure color to match the saturation of the sample 
color. Brightness is given by: 


B * / E(A)V(x)dX 

where E(x) is the spectral distribution of the sample and V(x) is the photopic response function of the 
eye, as explained in the section dealing with luminance. The measurement of the dominant wavelength and 
purity will be the subject of this section. 

The term color contrast is often used to describe the difference in visual sensation experienced when 
looking at two adjacent surfaces of different color. Experimental psychologists in this field use the same 
term to describe a phenomenon experienced when subjects observe a small gray area surrounded by, say, a 
large red area. Under such conditions, a greenish tinge is seen at the red-gray boundary. This effect is 
not thought to be of significance in visual systems. The term color difference appears to be preferred to 
color contrast to describe the former visual sensation. The CIE Technical Committee No. 1.3 has prepared a 
report (Ref. 42) on color difference equations which attempt to provide a measure for perceived color dif- 
ferences. It is assumed that these equations will enable a set of numbers to be calculated which could be 
used to describe a color picture in a way similar to the way contrast ratios are used to describe a black- 
and-white picture. 

3.5.2 Measurement of color 

In color television^ an additive process of color mixing, using three primary colors, red, green, and 
blue, is used to produce what the human visual system perceives as various colors across most of the visible 
spectrum. Standards for the specification of color were adopted in 1931 (some time before the development 
of color television) by the International Commission on Illumination (Commission Internationale d'Eclairage, 
or CIE). These standards set the red primary at a wavelength of 700 nm, green at a wavelength of 546.1 nm, 
and blue at a wavelength of 435.8 nm. The internationally recognized system of color measurement is called 
the CIE XYZ systen. It is based on the trichromatic theory of color vision, which states that any mono-' 
chromatic color is equivalent to the algebraic sum of three reference sources of light. 
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The development of a color-matching and specification system involved extensive testing using many 
subjects. These tests resulted in what are known as the tristimulus values for color mixing. Curves of 
the power spectra of the standardized primaries are shown in Figure 40. The red primary values are shown 
in curve X, green in curve and blue in curve 2. 

The tri stimulus curves contain only the information required to determine the amount of each primary 
color necessary to match any saturated spectral color. The information necessary for matching desaturated 
colors is not provided. For the tristimulus color-mixing system to be practical, a more useful and complete 
means of specifying color is required. 

A three-dimensional diagram with the tri stimulus values plotted directly could be used to represent 
colors of different hues and saturations. However, introducing the three related quantities x, y, and z, 
known as the chromaticity coordinates , produces a diagram that Ties on a plane; it is more convenient to 
use. The chromaticity coordinates are defined as follows: 


X + y + z 



X + y + z 


z 


z 

X + y + z 


Since, from the above definition of the chromaticity coordinates, x + y + z - 1, the values of any two of 
these coordinates are sufficient to completely determine the chromaticity. The coordinates x and y are 
usually chosen and the y values of all possible colors plotted against the x values of those same colors 
comprise the CIE chromaticity diagram shown in Figure 41. 



WAVELENGTH, nm 
Fig. 40 Tristimulus curves 



Fig. 41 Chromaticity chart; wavelengths along the 
spectral line are in microns. 


For 'light sources .having a spectral distribution given by E(a), the equations for the chromaticity 
coordinates are generalized; that is, 

. 

^ X + Y + Z 

y ° X + Y + Z 

where 

X = / P(x)jE(x)dx 
Y = / P(X)-E(x)dX 
Z = / P(X)jE(x)dX 
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The coordinates x and y will give a point on the chromaticity chart inside the locus of the pure spectral 
colors, such as at P. White is given by the point D on the chromaticity chart having coordinates of 

X = 0.313 and y 0.329. This is the CIE illuminant 0, which seems to be the current standard. If a 

line is drawn from D through P to meet the locus of the spectral color at Q, the dominant wavelength 
will be given by Q and the purity by DP/QD. 

The accurate measurement of the chromaticity coordinates for a sample light source requires the accurate 
measurement of the spectral distribution of the light source. Monochromators, which can be obtained as 

accessories to photometers, allow the photometer to be used as a spectroradiometer. The resulting function 

E(x) can be used in the preceding equations to obtain the tristimulus values X, Y and Z. These are used 
to calculate the CIE coordinates x and y of the sample source. If an extremely accurate measurement is 
not required or if only the relative chromaticity values between two sources are required, a good photometer 
fitted with tristimulus filters is quite adequate. These filters approximate the CIE primary colors enabling 
the photometer to perform the integrations in the preceding equations to give the tri stimulus values X, Y 
and Z directly. 

Color is generated in a visual system by mixing three primary colors, usually red, blue, and green. 
Knowing the CIE coordinates of each of these primaries provides the user or designer with the available 
spectrum of colors. For example, shown in Figure 41 are the chromaticity diagrams for an actual CRT, the 
RCA 1908P22 which has become a popular CRT for full -color daylight visual systems. All colors within the 
triangle can be generated using a suitable mixture of the primaries. When viewed through infinity optics, 
this triangle will be shifted due to the spectral transmission of the optics and may vary as a function of 
viewing angle. 

The color of each primary should be measured at the center of each display and at selected points on 
the periphery of the display. The measurement should be repeated after a suitable time interval (24 hr) . 
to check for color stability. It will be necessary, however, to use a standard source to check the stability 
of the photometer. 

3,5.3 Color differences 


Workers in the field of colorimetry have been trying for a number of years to establish a method for 
predicting and measuring color differences. In 1964, the CIE recommended the use of a three-dimensional 
uniform color space diagram in which the rectangular coordinates U*V*W* were non! i nearly related to the 
CIE tristimulus values X, Y, Z. The distance between two given points in this space diagram represented 
the perceptual difference between the colors defined by these two points. 

In 1978, the CIE issued a report entitled "Recommendations on Uniform Colour Spaces, Colour Difference 
Equations and Psychometric Colour Terms" (Ref. 42). It appears that several techniques are being used for 
estimating color differences but the following method, which is a modification of the 1964 u*V*W* color 
space, is one of two uniform color spaces recommended by the CIE. It seems to be preferred when colored 
lights are mixed additively and, therefore, seems appropriate for CRTs and other types of television 
displays. 


The first approximately uniform color space is a modification of and supersedes the CIE (U*V*W*) space. 
It is produced by plotting in rectangular coordinates the quantities L*, U*, V* defined by 


L* = 116(Y/Yn)^/^ - 16 Y/Y^ > 0.01 


with 


u* = 13 L*(u' - u;,) 
V* = 13 L*(V - v;,) 


=)T .15%\3Z 
"n X, + 15 Yn + 3 Z, 


V - 9 Y 

^ ■ jT + 15 Y + 3 Z 

V. 9 Yn 

'^n X^ + 15 Y„ + 3 Z, 


The tri stimulus values Xp, Yp, and Zp define the color of the nominally white object-color stimulus. 
Usually, the white object-color stimulus is given by the spectral radiant power of a CIE standard illuminant. 
Under these conditions, Xn, Yp» and Zp are the tristimulus values of the standard illuminant with Yp equal 
to 100. 


The total difference aEuv between two colors each given in terms of L*U*V* is calculated from 
AEjv = [(AL*)^ + (aV*) 2 + (AU*^]l/^ 


The color space defined above is called the CIE 1976 (L*U*V*) space and the color difference formula 
the CIE 1976 (L*U*V*) color difference formula. The abbreviation CIE LUV is recommended. It is also 
recoimiended that anyone desiring to use color difference formulas read the CIE publications dealing. with 
the subject and keep abreast of any new developments. 


3.5.4 Use of color 


Whether color should be used in visual systems is a debatable question. There is little experimental 
evidence of its effect, and what evidence there is compares monochrome versus color in flight simulators 
without relating them to actual flight conditions. There is presently no substantial objective evidence 
either for or against the use of color in visual flight simulators (Refs. 43 and 44). Even in well-designed 
simulator-versus-flight studies to determine the effect of color in simulation systems, it will be difficult 
to hold constant other visual characteristics that may affect the results more than the monochrome versus 
color characteristic. 
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The main objection to using color systems stems from the lower resolution available with color displays. 
The highest resolution color television display has significantly lower resolution than many monochrome 
displays, and it is probable that this situation will always exist. The lower resolution is usually due to 
the existence of three independent channels providing the red, blue, and green components which must ulti- 
mately be registered at a single image plane. Even display devices that do not use independent color channels, 
such as the GE light-valve or field-sequential displays, have an inherently lower resolution than their mono- 
chrome counterparts. Night and dusk calligraphic visual simulation systems in which beam-penetration CRTs 
are used generally outperform in resolution all other color systems but still have registration problems, 
particularly at the edge of the display. Their color range is of course very limited (see Fig. 41). 

The range of colors produced by most display devices compares well with the range of available printing 
inks and photographic dyes. Figure 42 (Ref. 7) relates the color gamut of one of these tubes to colors in 
human experience. The range of this tube also covers the portion of the chromaticity chart where the eye is 
most sensitive to small differences in color. This sensitivity is indicated by the MacAdam ellipses (Ref. 45), 
shown in Figure 43, where the smallest ellipses are indicative of the greatest human color sensitivity. 



AXES OF ELLIPSES ARE 
10 TIMES THE STANDARD 



Fig. 43 Chromaticity matching under ideal conditions 
The MacAdam ellipses illustrate the adjust- 
ment precision possible when chromaticity 
matches are made by a single observer under 
ideal conditions. 


Most observers, whether they are pilots, visual simulation system engineers, or armchair football 
players, prefer a color scene to a black-and-white scene, simply on subjective grounds. Such subjective 
opinions cannot be dismissed because they might well provide the information that would come from the appro- 
priate experiments. Despite the increased cost and lower resolution of color in visual simulation systems, 
there are factors that argue for its use, including detection, recognition, display aids, key-mixing, and 
special tasks. Each is discussed below. 

An important aspect of flying, particularly in military applications, is the detection of objects on 
the ground. Color plays a part in this process. Both flora and fauna use color either for concealment or 
to attract attention; the same principles apply to the concealment or recognition of ground objects viewed 
from an aircraft. However, in the simulator, the poor resolution inherent in all visual displays does not 
permit objects to be detected by the pilot at the same distances as in real life. To compensate for this 
effect, the simulated object can be colored in a manner to allow detection at ranges more closely approx- 
imating the real-world detection ranges. 

An object is more easily recognized in color than in monochrome; for example, a photograph of the con- 
tents of a bowl of fruit will be recognized much more quickly in color than in black and white. Similarly, 
in a projected image of ■ the sky and the ground, if the sky is blue and the ground is brown/green, the pilot 
will rarely mistake his orientation, even in violent combat maneuvers. In order that their color be cor- 
rectly perceived, the sizes of objects must be greater than 10 arcmin, or about an order of magnitude greater 
than the size at which the object could be detected from spatial information alone. This indicates that 
color will not aid in flight tasks requiring air-to-air and air-to-ground target detection. However, in a 
visual simulation systan with resolution capability that is not as good as that of the eye, color can provide 
information to aid in recognition and identification of targets. In such visual simulation systems, the color 
of a target as it is approached becomes apparent before there is enough spatial detail depicted for its recog- 
nition. According to Johnson’s criteria, a target must subtend 13 TV lines if it is to be detected reliably, 
recognized, and identified. In a system with 2 arcmin per TV line, a target subtending approximately five 
TV lines would be large enough (i.e., approximately 10 arcmin) for its color to be correctly perceived. This 
indicates that the color of the target would be perceived at one-half the size (or twice the range) of that 
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required for recognition in the absence of color through spatial information alone. In an air-to-air com- 
bat visual system with resolution capabilities below the capabilities of the eye, target aircraft will not 
be identifiable at ranges as great as those in a real-world situation. It may be possible in such a visual 
system to color-code targets representing different aircraft so that they might be identified at ranges 
equivalent to real-world ranges. The color in this case would be an artificial cue substituting for the 
lack of spatial resolution that would be necessary for identification at equivalent ranges. 

Many military aircraft and some civil aircraft employ head-up displays superimposed on the outside 
world. Such displays are colored in a manner to help them to be observed against the background of the 
visual scene. To maintain at least a semblance of the real situation (light levels are grossly degraded), 
it is preferable to present the correct colors both on the HUD and in the visual scene. 

Simulation is an illusion, and the simulator engineer must create the illusion. Television-based 
equipment plays a large part in creating the illusion in most simulators, in the form of image producers. 
The advent of color TV opened many new possibilities for special effects, based on video mixing and color 
keying. In striving for realism, it is permissible to offset a deficiency in one aspect by an enhancement 
in another — for example, by exaggerating the size of an object at long range. Similarly, color can be 
used to exaggerate some aspects of the scene and to highlight others, so that the pilot receives a more 
realistic impression. At both an artistic and a technical level, the use of color gives more scope for 
ingenuity in producing the illusion. Another area where color may be very beneficial is in displaying 
computer-generated texturing. Texturing, under conditions of motion, can cause severe scintillation, espe- 
cially if the contrast is not greatly attenuated. Proper correction of this scintillation through anti- 
aliasing hardware is very expensive. One possibility for minimizing the scintillation problem without 
major investments in image generation hardware is to use color contrast to introduce the texturing rather 
than luminance contrast. 

Certain special tasks cannot be performed on monochrome visual systems, such as an aircraft carrier 
landing using a Fresnel Lens Optical Landing System (FLOLS) or an airfield landing using Visual Approach 
Slope Indicator (VASI) lights. 

The perception of color in the peripheral vision is considerably different from its perception in the 
fovea 1 viewing area. This is due to the increased proportion of rods to cones away from the fovea and the 
fact that the rods are more sensitive to the blue end of the spectrum. The macular pigmentation in the 
central 10° of the retina is slightly yellow and is also a factor. Table 3 is taken from a paper by 
Kinney (Ref. 46). It shows the relative apparent brightness of small colored targets in the peripheral 
vision. Blue objects appear considerably brighter in the peripheral region, which suggests, perhaps, that 
wide-angle visual systems using beam-penetration type CRTs having no blue content may not give particularly 
good peripheral cues. 


CONDITION OF VIEWING 

BLUE 

GREEN 

YELLOW 

RED 

Foveal 
2° target 

1.0 

1.0 

1.0 

1.0 

10° in periphery. 





2° target, 

3 cd/m^ surround 

5.5 

1.3 

0.5 

0.5 

40° in periphery, 
2° target, 

Scotopi c 

7.4 

1.3 

0.1 

0.01 


TABLE 3: Relative apparent brightness of color targets 

These and other aspects of color vision seem to suggest that color should be included in a visual 
simulation system. In the absence of controlled psychophysiological experiments, our lack of knowledge of 
the human visual system suggests that simulator visual systems should resemble the normal operational 
environment as much as possible. Color could be considered an important visual cue in daylight scenes if 
this premise is accepted. 

One further aspect of color concerns the use of very narrow-band spectral sources such as lasers in 
visual systems. Fincham (Ref. 47) found that subjects used the color fringes around an out-of-focus image 
due to chromatic aberrations within the eye as a cue for accommodation. Campbell and Westheimer (Ref. 48) 
confirmed this result, finding that many subjects were unable to focus on a monochromatically illuminated 
object. Other subjects were unaffected and seemed to use the blurring of the image caused by spherical 
aberrations as a cue. Most subjects who were affected by the monochromatic lights were able to use other 
cues for accoimodation after a brief period of training. It is not known if the preceding remarks apply 
to scenes generated using three monochromatic light sources. Even if the problem does exist in such sys- 
tems, the ability of the eye to adapt may prevent any reduction in the value of the system for training or 
research simulators. 

The only use of color as a depth perception cue is due to the selective absorption and scattering of 
light by the atmosphere. This process gives a bluish tinge to distant objects as well as making them less 
distinct. The cue is known as aerial perspective. Just how much the blue shift aids in estimating range 
is not known, but it can be simulated fairly readily on most CGI systems and probably adds to the overall 
realism. Color stereopsis in pilots has been investigated by Kraft and Shaffer (Ref. 21) who found that 
many pilots would place colored objects closer or farther away from their actual positions, depending on 
the saturation of the color and the type and extent of the eye defect. 

Since color does not appear to be essential for air-to-air combat, but does appear to be desirable for 
air-to-surface simulation, a color system less sophisticated than a full, three-primary-color system might 
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be worth considering. A two-color red and green system could cover the green and brown range of hues 
typically found in natural terrain, but the sky, of course, would have to be some color other than blue. 

The cost reduction may be greater than proportional to the number of colors because some problems, such as 
color registration, would be much simpler to resolve in a two-color than in a three-color system. 

3.6 Noise 

3.6.1 Definition 

The most general definition of noise is any intensity variation existing in the displayed image that 
did not exist in the original scene. There are two basic types of noise: coherent and random. Random 

noise can usually be described in terms of statistical fluctuations; it is random both spatially and tem- 
porally. Coherent noise usually has a fixed spatial and temporal relationship to the desired image, such 
as the fixed chicken-wire pattern produced by a fiber optic faceplate or the interference effects caused by 
an electronic signal synchronized to the field rate of the display. Moving patterns such as those caused 
by microphonic components or nonsynchronous electronic interference are also called coherent noise, providing 
they can be seen as distinct patterns by the human visual system. The speckle® patterns produced by laser 
displays are also a form of noise and may be coherent or random, depending on the technique used for reducing 
the speckle. No further mention will be made of speckle; however, if laser displays become widely used in 
visual simulation systems, a standard technique for measuring speckle should be specified. 

The random noise seen on model -board systems using a closed-circuit television is invariably produced 
in the television camera. The nature of the noise may be photon noise, shot noise, preamplifier noise, or 
other noise-producing processes within the camera tube itself. The reason for the noise is the statistical 
variation in the number of photons or electrons that define an element of the scene in the sampling time of 
the sensor. If, for instance, a picture element on the face of the television camera tube receives an 
average of N photons each field time, it can be shown that the root-mean-square fluctuation is equal to 
v^. The signal -to-noise ratio will therefore be N/yTI or This is the theoretical limit for the 

signal-to-noise ratio which will invariably be degraded by other processes further down the video chain 
(see Ref. 49). 

Random noise as described above is insignificant in CGI visual simulation systems, but coherent noise 
can be a problem. 

3.6.2 Measurement of noise 

Because an insignificant amount of random noise is produced in the display electronics, a valid mea- 
surement of signal-to-noise ratio can be made at the output of the TV camera. Care should be taken, however, 
that the video signal being measured either has a unity gamma or that the measurement is corrected for the 
actual gamma. 

Several instruments are available for measuring the signal-to-noise ratio on commercial TV systems, 
using normal line and field rates. The technique usually consists of placing a white card in front of the 
camera; the card has a uniform luminance equal to the normal peak luminance. The video signal at the out- 
put of the camera is then sampled at the horizontal line frequency and the noise signal measured. By mea- 
suring the peak signal level relative to the black level, the signal-to-noise ratio is determined. Unfortu- 
nately, these instruments tend to be expensive and usually cannot be used with the nonstandard TV systems, 
which are often used in visual simulation systems. 

Some TV equipment manufacturers have successfully used a technique in which the type of dual -beam 
oscilloscope that is ordinarily available at most simulator sites is utilized; it is described below (see 
Ref. 50 for a full description). The camera is set to look at a standard resolution chart and the video 
output is displayed on two channels of an oscilloscope; the two channels have identical gain. The oscillo- 
scope is synchronized to display a single horizontal line of video which passes through a broad white 
vertical bar on each side of which there are broad black bars. The vertical shifts on each channel are 
adjusted so that the two waveforms are exactly overlapping and then they are gradually separated until a...... . 

distinct dark line can just be seen in the overlapping noise on the white bar. The vertical separation can 
be shown to be equal to twice the rms noise. With a little practice, repeatable measurements can be obtained 
with little difference among observers. The measurement should also be repeated on the black bars. The 
signal-to-noise ratio in both cases should be relative to the peak white signal. 

The measurement of coherent noise in the video signal Is not recommended. Noise signals that are 
clearly visible on the display may be imperceptible in the video because of random noise. The preferred 
technique would be to measure the contrast between the noise signal and the adjacent area on the display, 
using a photometer (described in Sec. 3.3). The measurements should be performed on both black-and-white 
areas of display. Because of the spatial integration processes within the human visual system, the mea- 
surement may not be particularly meaningful. The visibility of coherent noise patterns is highly dependent 
on their shape and size and subjective measures (e.g., imperceptible, perceptible, distracting, highly dis- 
tracting) may be preferable. 

3.6.3 Effect of noise 

Two quite distinct effects of random noise are relevant to visual simulation. The first is the objec- 
tionable effect that noise has on picture quality, ‘and the second relates to the degree of difficulty in 
performing a given visual task in the presence of noise. The two effects may be related but the relationship 
is not obvious. A certain amount of work was done in the 1950's and 1960's to determine acceptable noise 
levels for broadcast television. Barstow and Christopher (Ref. 51) positioned a number of observers at a 
distance equal to 4 times the picture height and asked them to judge the picture quality while various 

®When a laser light, scattered by one side of the object, interferes with the light scattered by the 
other side of the object, a random pattern of interference fringes is generated due to a laser's high coher- 
ence characteristics. This random interference pattern is called "speckle." 
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amounts of noise were added electronically to the video signal. The results are shown in Figure 44. The 
two curves are for white noise, with a constant power over the bandwidth of the display (4,5 MHz), and for 
low-frequency noise, with a cutoff at 200 KHz. As can be seen, observers can tolerate 2-3 dB lower signal- 
to-noise ratio for white noise than for low-frequency noise. 


Some television systems have noise that is pre- 
dominantly high frequency and gives a fine-grained 
appearance to the picture. This type of noise is 
even less objectionable than white noise. 

As stated earlier, random noise is due to the 
statistical fluctuation in the signal during the 
sampling time of the sensor. Of course, the human 
visual system is the final signal processor and its 
integration time must also be considered. It is 
difficult, if not impossible, to measure this accu- 
rately, but most workers in the field of target- 
detection theory use values varying from 0.2 sec at 
very low light levels to 0.05 sec for very high light 
levels. This relationship between luminance and the 
integration time of the human visual system seems to 
be generally accepted, and Legault (Ref. 52) suggests 
that the integration time is approximately equal to 
luminance raised to the minus x power, where x is 
nominally 0.1. This relation would give an equiv- 
alent increase in signal-to-noise ratio of 1 dB 
(which is barely detectable) for a decrease in lumi- 
nance by a factor of 10. It does not seem to explain 
the rather obvious decrease in picture noisiness that 
occurs when the luminance of a definitely objection- 
ably noisy picture is decreased by a factor as low 
as 2, using neutral density filters. 


EXTREMELY 

OBJECTIONABLE 


DEFINITELY 

OBJECTIONABLE 


SOMEWHAT 

OBJECTIONABLE 

IMPAIRMENT 
BUT NOT 
OBJECTIONABLE 

DEFINITELY 
PERCEPTIBLE, 
ONLY SLIGHT 
IMPAIRMENT 
JUST 

PERCEPTIBLE 

NOT 

PERCEPTIBLE 



60 55 50 45 40 35 30 

WEIGHTED NOISE MAGNITUDE 
(dB BELOW IV P-P) 


Fig. 44 The aesthetic opinion ratings of broadcast 
signals impaired by video (base band) 
weighted values 


The integration time of a TV camera is usually taken to be equivalent to its field time, that is, con- 
siderably less than the nominal integration time of the eye at normal display luminances- A scene having a 
given signal-to-noise ratio (measured in the video signal) will, therefore, appear slightly less noisy in a 

60-Hz TV system than in a 50-Hz system. The difference is much more obvious (equivalent to 5 dB) when a 

field sequential TV system operating at 180 fields per second is compared with a standard system operating 
at 60 fields per second. 

The second effect of noise is to reduce the ability of an observer to perform a given visual task. 

The effect can be summarized by referring to Figure 45, which shows a typical MTFA area bounded by the sys- 

tem MTF and the observer threshold detectability curve. Noise tends to elevate the threshold detectability 
curve and to reduce the MTFA. The numerical change in MTFA may not be particularly significant and may be 
comparable to errors in the measurement of the MTF; it may even be insignificant when dynamic effects are 
considered. Nevertheless, the figure indicates that at low spatial frequencies noise reduces the maximum 
contrast, and this can be the most significant effect on visual simulation systems using vidicon cameras. 

At the higher spatial frequencies, however, noise can prevent a pilot from distinguishing the threshold 
stripes at the appropriate range or from detecting a target in an air-to-ground attack mission. It may 
also detract from a pilot's ability to detect subtle changes in the image of his adversary during air-to-air 
combat. 


Although MTFA is a good measure of picture qual- 
ity, it should be used with care when trying to pre- 
dict a pilot's performance for a given visual task. 

In particular, it should be noted that the threshold 
detectability curves are usually obtained under con- 
ditions where the observer is allowed a long time to 
detect a target or to discriminate a test object; 
moreover, the observer may also be allowed to change 
the magnification of the test scene. The MTFA also 
fails to predict the ability of a television system 
to show details as fine as a single human hair. The 
bandwidth of the television system may be an order 
of magnitude less than one would expect necessary to 
pass such detail and, yet, single human hairs can be 
clearly seen on broadcast TV. The reason is 
obviously due to the human visual system's ability 
to integrate along a line and the limiting parameter 
seems to be the signal-to-noise ratio. 
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Fig. 45 Effect of noise on MTFA 


Vernier and stereoscopic acuity are probably affected in a similar manner by signal-to-noise ratio. 

Coherent noise is usually distracting and should obviously be reduced as much as possible. When the 
coherent noise pattern is uniform over large sections or over the entire display, such as the triad struc- 
ture in a shadow-mask CRT, aliasing occurs (described in Sec. 4.5; see also Sec. 4.2.3 on noise from the 
point of view of temporal properties in scene generation). 
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3.7 Visual Simulation System Characteristics at the Component Level 
3.7.1 Introduction 

The discussion in this section has focused on visual system light energy parameters and measurement 
techniques in general for visual simulation systems. These parameters are equally applicable at the com- 
ponent level of the visual system; however, there are other characteristics of light energy that must be 
considered when working at this level. This section will discuss some of the unique light-energy charac- 
teristics associated with various components of a visual simulation system and with the interfaces among 
these components. It will also address the manner in which the performances of individual components can 
be combined to predict the overall performance of a total visual system. 

When considering light-energy parameters at the component level, the four basic characteristics - 
luminance, contrast, resolution, and color — must be dealt with. Also, some means of predicting the overall 
visual simulation system performance in terms of the four basic parameters based on individual component 
level performance is required. For each of these four parameters, the following will be discussed: (1) the 

characteristics and parameters that must be considered at the component level, (2) the technique for comput- 
ing overall visual system performance based on component level performance, and (3) an example showing the 
computation of overall system performance. 

The samples presented are typical examples of a single calculation of, in these cases, on-axis system 
performance. To characterize the performance of any visual system completely, component measurements and 
system performance calculations must also be made off-axis throughout the field-of-view. In most cases, 
component and system performance will tend to degrade as the viewing angle departs from on-axis. 

There is a wide variety of types of visual simulation systems that have been conceived and developed 
over the past 25 to 30 years, and many of these are in use today. Some are fairly simple in concept; how- 
ever, most employ a rather complex arrangement of diverse component parts. Nevertheless, most of these 
simulator visual systems can be considered to be made up of three basic types of components: storage 

devices, converters, and relays. Figure 46 shows an arrangement of five components that are found in most 



Fig. 46 Visual system major component parts 

visual systems. Typical storage devices are physical scale models, transparencies, photographic film, and 
computer memory. Most simulator visual systems use two different types of converters. The first type con- 
verts the stored image into a television video signal or into light energy containing information about the 
stored image. The second type converts a television video signal into light-energy information about the 
visual image. Probe/television cameras, laser scanners, flying spot scanners, and computer generators are 
converters that convert the stored image into television signals. Illuminating systems that utilize trans- 
parencies or film are converters that convert the stored imagery into light energy representing the imagery. 
CRTs, television projectors, and laser projectors are converters that convert television video into light 
energy containing information about the visual image. 

There are two basic types of relay components: one that relays television video and the other that 

relays light. The television relay component is the video amplifier and associated electronics. Some video 
processing in the form of raster mapping or insetting may also be involved. Optical relays and processing 
are used in some visual simulation systems. The variable anamorphic film system is an example in which 
optical relaying and processing, using anamorphic optics, are used to present the imagery for an instantaneous 
view point off the nominal film viewpoint. The final system component is usually a relay device that relays 
the light to the pilot/observer. These devices are typically projection screens, infinity optics, or true- 
view optics. The screen presents the Imagery at a fixed distance, usually 3-6 m from the pilot, the infinity 
optics present the imagery near infinity, and the true-view optics present the imagery at distances that 
various parts of the imaqery would actually be in the real world. An example of a true-view optical system 
(described in Sec. 2.4.3) is a visual system developed for simulating a refueling boom in an aerial refuel- 
ing trainer that was built recently for the U.S. Air Force. 

Figure 47 shows some of the typical components of a visual simulation system and some of the combina- 
tions of these components that have been employed in various visual systems. Most simulator visual systems 
nicely fit this component parts definition. For example, a model-board/television system with a real -image 
screen display would be composed of components lA, 2A, 3A, 4B, and 5A; a CGI system with an infinity display 
would be composed of components ID, 2E, 3A,'4A, and 5B. A few systems are more direct from image storage to 
image display and do not have an image relay/processor or a second converter, for example, a point-light 
transparency system and a true-view model system. Nevertheless, most visual simulation systems can be 
thought of in terms of these five visual system component parts. The first three for a particular simulator 
visual system are usually lumped together and referred to as the image generator and the last two as the 
visual display. Some visual simulation systems may enploy more than one basic visual system concept; for 
example, some air-to-air combat visual systans use a point-light-source/ transparency system for the Earth/sky 
background and a model/television system for the target aircraft. 
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STORAGE CONVERTER RELAY/ CONVERTER RELAY 



Fig. 47 Typical visual system components and some of the various combinations 


3.7.2 Luminance 

In most visual simulation systems in use today, the light energy that enters the eye of the viewer 
originates in the image input device, which is usually either a CRT or some type of projector. The dis- 
played image, if viewed directly, is a real image on the CRT face or a projection screen, or, if viewed 
through some form of collimating optics, is the virtual image of the CRT face or projection screen. 

In a display system composed of a light-valve projector, a screen, and collimating optics, some of the 
following characteristics of light at various points in the display system must be considered: 

1. Polarization of the light: Some light-valve projectors, for example, liquid-crystal, light-valve, 

and Titus valve, produce linearly polarized light. If such a projector is used with optics that employ 
polarizing elements, for example, in-line infinity optics (tradename Pancake Window), the axis of polariza- 
tion must be known to ensure maximum transmission through the optics. A photometer with a rotatable polariz- 
ing filter can be used to determine the axis of polarization. 

2. Directional characteristics of the light: Various screen materials can produce reflected or trans- 

mitted light with widely different directional characteristics. Ordinary projection screens reflect only a 
portion of the incident luminous flux toward the viewer, whereas a specular surface can reflect all of the 
incident light toward the viewer, provided that he is placed exactly on the axis of the reflected ray. 

Special semi specular screen material can be used to increase the amount of light reaching the viewer. These 
screens are considered to produce light gain, which is defined as the ratio of the on-axis luminance of the 
semi specular screen to the on-axis luminance of a diffuse screen having the same illumination. The price to 
be paid for this gain in lumiijance is the more rapid fall off of luminance as the viewer moves away from the 
viewing axis. The traditional reference matte white surface used for determining the gain of other screen 
material is a block of magnesium carbonate. Figure 48 (from Ref. 53) shows test data from a magnesium car- 
bonate screen. Figure 49 (from Ref. 53) shows the relative amplitude of reflected light for a glass-beaded 
screen. The on-axis reflected light is approximately 4 times greater than the reference magnesium carbonate 
screen; thus, it has a gain of approximately 4. The exit pupil for this screen is 16°. The term exit pupil 
is used to describe the distribution of reflected light and is defined as the angle over which the luminance 
level of reflected light is equal to or greater than one-half the on-axis luminance level (50% of maximum). 
Figure 50 (from Ref. 53) shows the relative amplitude of reflected light for a semi specular screen that has 
a gain of 43 and an exit pupil of 10°. Figure 51 (from Ref. 53) shows the relative amplitude of reflected 
light for a very high-gain, retro- reflective screen; its gain is 688 and its exit pupil is 1°. 

3. Bandwidth and transmission characteristics: The spectral characteristics of the input light must 

be selected to match the transmission characteristics of the optics. This requires careful selection of 
phosphors for CRT's and illuminating sources and filters for projectors. The overall transmission efficiency 
of a visual display system is simply the product of the transmission efficiencies of the n individual 
components, 

■^total = T, X X T, . . . X 
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SCREEN: MAGNESIUM CARBONATE 
TEST CONDITIONS: 

PLANE - HORIZONTAL 

INCIDENT ILLUMINATION - NORMAL 



SCREEN: GLASS-BEADED 


TEST CONDITIONS: 

PLANE -HORIZONTAL 

INCIDENT ILLUMINATION - NORMAL 



SCREEN: SEMI-SPECULAR 
TEST CONDITIONS 

PLANE - HORIZONTAL 



SCREEN: RETROREFLECTOR 
TEST CONDITIONS: 

PLANE -HORIZONTAL 

INCIDENT ILLUMINATION - NORMAL 
Q 



Fig* 51 Test data, retro-reflective screen 


As an example, consider the following display system consisting of a light-valve projector, a rear projec- 
tion screen, and a set of in-line infinity optics. The projector produces about 240 m of linearly polar- 
ized light projected onto a 0.5-m^ screen with a gain of 8.0, the polarization axis alignment factor is 0.7, 
and the transmission efficiency, T, of the optics for linearly polarized light input is 0.02. . .. 




OPTICS 



The system brightness at the viewing position is computed as follows: 


Bj = ^ (o) 


that is 


Bg = 17.11 cd/m^ (at the pilot's eye) 
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3.7.3 Contrast 


Although the light energy that enters the eye of the viewer originates in the image input device, com- 
ponents other than the image input device, screen, and optics can affect the overall system contrast per- 
formance capability. Both the image pickup (optical probe) and the image processing/relaying (camera/video 
chain) for a model /television system can affect the overall system luminance contrast. For high-contrast 
ratios (B 1 /B 2 ), the overall visual system contrast capability is affected by the contrast capabilities of 
the n individual components approximately in accordance with the following expression: 


1 

^total 



For example, consider a visual system as before 
composed of a light- valve projector, screen, and col- 
limating optics driven by a computer image generator. 

In this particular case, the primary components 
affecting total system contrast capability are the 
projector, with a contrast ratio of 30:1; the screen, 
with a contrast ratio of 250:1; and the optics, with 
a contrast ratio of 30:1. The overall system con- 
trast ratio is computed as follows: 

r -1+1+1 
^total " M 2 B 0 

that is, 

"total * 



PILOT 


3.7.4 Resolution 


The overall system resolution is affected by the resolution capabilities of each component in the visual 
system. As described earlier, the most widely accepted method of measuring visual system resolution is the 
MTF technique. MTF is a measure that takes into account not only the spatial frequency that can be depicted 
but also the contrast capability as a function of spatial frequency. In layman's terms, the MTF of a com- 
ponent indicates the capability of that component to transfer contrast through the component as a function 
of spatial frequency. The overall system MTF is simply the cascaded MTFs of the n components. 

MT^otal = MTF, X MTF, x . . . x MTF^ 


Consider, once again, a visual system composed 
of a light-valve projector, screen, and collimating 
optics driven by a computer image generator. When 
computing the system resolution, the percent modula- 
tion of the component at the system spatial resolu- 
tion design goal is usually used rather than the MTF, 

The computer image-generation system usually has a 
picture element size chosen to meet the system spa- 
tial resolution specification. In this case, the 
spatial resolution is assumed to be 4 arcmin per TV 
line. A computer image- generator without edge 
smoothing is assumed to produce a lOOX modulated video signal at 4-arcmin resolution. (However, this is not 
true for a CGI system with edge smoothing, which reduces the percent modulation at high spatial frequencies.) 
The video amplifier has a bandwidth of 40 MHz and has a 95^ modulated video output at 4 arcmin. The percent 
modulation for the projector is 60 and the percent modulation for the optics is 80. The screen is assumed to 
have a negligible effect on totai system MTF. 



that is. 


% modulation^^^gl = 0.95 x 0,60 x 0.80 x 100 
% modulation^Q^^ = 45.6 at 4 arcmin 


3.7.5 Color 

Most color visual systems in use today are based on the psychophysical function of the human eye that 
equates monochromatic color to the sum of various fixed reference colors at various luminance levels. There 
are so-called "full -col or" systems in which three reference colors (usually red, blue, and green) are used, 
and two-color systems in which two reference colors are used; the two-color systems can represent a range of 
colors between the two reference colors. Color measurements at the component level are with respect to the 
reference or primary colors. In CRTs, the primary colors and the range of colors that can be represented 
are determined by the emission characteristics of the phosphors. In light-valve projectors, the colors are 
determined by the emission characteristics of the lamp and the filters selecting the primary colors. 

If the CRT or projected images are viewed through collimating optics, the transmission characteristics 
of the optics over the spectrum of the primary colors must be considered. If the primary colors are not uni- 
formly transmitted, a color shift will result. However, this can usually be compensated for by adjusting 
the luminance levels of the primary color inputs. The classical mirror-beam- splitter, reflective optical 
systems are usually broadbanded and do not introduce significant chromatic aberrations. In-line infinity 
optics, which are reflective optics, are relatively free of chromatic aberrations; however, they are not. 
broadbanded (because of elements, such as polarizers and quarter-wave plates, that tend to peak their trans- 
mission in the green part of the spectrum). Refractive optics are fairly broadbanded, but introduce sig- 
nificant chromatic aberrations off-axis at the extremes of the field of view. 
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4. TEMPORAL PROPERTIES 
4.1 Introduction 

The measurement of temporal properties of flight-simulator visual systems is confounded by the variety 
of techniques used for the production of the visual scene. Each of these techniques results in peculiar 
difficulties of measurement. 

The simplest of the various systems is the Earth/sky shadowgraph projector, for which the content of 
the data base is trivial and all the temporal properties of the system are contained within some form of 
servomechanical drive. The directly viewed, optical projection of the scene contains no temporal properties. 
The fact that the light source is not collocated with the viewpoint may cause distortions but this is a 
static property. 

Film-based systems, too, usually have a direct, optical display (they may use a flying-spot scanner/TV 
presentation), but in this case the data base consists of a series of "views" that are presented without 
optical distortion (provided the taking and projecting lenses are of the same focal length) only along a 
unique flightpath. In general, the dynamics for a given maneuver depend both on the form of the maneuver 
and on the portion of the total scene being considered. 

Both computer-generated imagery (CGI)® and model -board systems depend on an electronic video link ter- 
minating in a TV or projected type of display. In these cases, there are temporal properties associated 
both with the manipulation of the scene generated from the data base and with the video link itself. 

The Ideal measuring location for these temporal properties is at the viewpoint, that is, at the pilot's 
eyes (or more strictly, midway between them). The next best location is at the surface of the viewing screen. 
There are difficulties associated with both of these locations, though a possible means of measuring some 
overall system properties is outlined in Section 4.4. Failing that, the system must be split into two, con- 
sidering separately the dynamics of the scene generation and those of the video link. All those elements of 
the complete visual system that are deliberately manipulated to give the illusion of movement of the simulated 
aircraft are contained within the "scene generator." The "video link" is specifically not restricted to its 
television connotation, but includes all elements of the system associated with the visual pathway. In other 
words, it includes not only the television video channel, when appropriate, but also any optics or light 
sources, while excluding the servomechanisms that may drive stjch elements. As stated above, in shadowgraph 
systems the complete dynamics are contained within the scene generator, and any measurements are consonant 
with similar measurements on model-board systems. In the latter case, fortunately, the frequencies of 
interest are very different from those of significance in the video link, which are themselves well above 
the normal frequencies involved in aircraft operations. 

Although servomechanical systems are generally characterized by a performance dependent on frequency, 
the discrete nature of systems based on CGI produces fixed time delays. Also, the absence of the smoothing 
that is produced by the camera of model -board systems produces many unusual dynamic characteristics in the 
perceived computer-generated image. These are described in detail in Section 4.5. 

Time lags are discussed in Section 4.2.2 as they occur in the scene generator. However, there is a lag 
that is associated with the display that is not covered in that section. The readout process of most if not 
all television camera image tubes cannot completely discharge the image storage mechanism of the tube in a 
single scan. The capacitive nature of the target and the resistance of the reading beam of electrons usually 
result in an exponential decay of the image which may take several scans before the initial image is no 
longer visible. This effect is called lag and reduces the dynamic resolution of a visual simulation system. 
Excessive lag will cause smearing of a moving image and after images may also be visible. For tubes operated 
in the normal 2:1 interlaced mode,’ lag is specified as the percentage of the signal remaining on the third 
field after an illumination source is removed. The signal would normally be that due to a bar chart and the 
illumination would be removed immediately before the first field so as to allow a full field for the image 
to be integrated on the storage element, that is, target or photoconductor. Values may be as low as 2% or 
3 % for a low-lag tube, such as a Plumbicon, oras high as 20% for certain vidicons. The latter devices have 
extremely poor dynamic resolution, even. at relatively low image velocities. Although the measurement of this 
imaging lag is fairly straightforward, a more meaningful measurement to a user is dynamic resolution, which 
is described in Section 3.4. 

It remains only to define the metric. Since the essential contribution of the eye is the identification 
of position, measurements should be based on this. However, many dynamic characteristics of visual systems 
are a function of velocity; therefore, velocity must be used as the input where appropriate. Finally, it 
is necessary that the visual display at least be able to match the dynamic performance of the aircraft in 
the tasks to be simulated, so that maximum capabilities in acceleration and velocity (suitably scaled) of 
the visual simulation system also need to be measured. However, all measurements are preferably made in 
terms of visual system dimensions, and are not scaled (by the scale of the model) to "real-world" equivalents. 
This avoids the presentation of multiple tables for the linear translations where several scales of models 
are used on the same basic system. Moreover, to obtain the real-world values, it is only necessary to 
multiply these values by the appropriate scale factor, and such scale factors as are normally available at 
the installation should be quoted. 


®There are two general types of computer image generation. The raster-scan type builds the image on 
the display device by moving the writing spot, moving in a series of horizontal lines in the same fashion 
as conventional television. The writing spot is modulated in luminance to portray the image. In the 
calligraphic type of image generation, the display spot is moved between any two points on the display 
device. As the spot moves, it draws a line between the two points, which usually represent the endpoints 
of a line. The spot is then blanked as it moves to the initial point of the next line. 

’Double interlaced scanning is the system of scanning alternate scan lines (all odd lines first and 
all even lines second). Each set of lines becomes a field; the fields, when displayed, are superimposed 
to create a frame or complete picture. Used to reduce the problem of flicker (see Sec. 4.3.1). 
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The input signal should be inserted as “close” to the visual simulation system as possible, that is, 
uncoupled from the aircraft model and associated computation. However, where an essential loop closure in 
the normal drive to the visual simulation system is within the host computer, this should be retained, for 
it is generally incorporated to improve system performance. Concomitant degradation due to digital-to- 
analog or analog-to-digital conversion, which may thereby be encompassed, forms part of the overall system 
performance. 

Measurements of several of the characteristics identified in this section are by analysis, using 
discrete Fourier transforms, of the system output for sinusoidal inputs. A detailed description of this 
technique is given in Appendix C. 

4.2 Scene Generation 

4.2.1 Excursion limits 

4.2. 1.1 Definition - Excursion limits describe the maximum capability of the system, determined by 
the extent of the data base, and the capability in the translational and rotational freedoms of movement. 
The limits may be imposed by displacement, velocity, or acceleration capabilities. There are two kinds of 
1 imits: 


1. System limits of the simulated scene generator are the extremes of displacement, velocity, and 
acceleration that can be achieved during single-degree-of-freedom operation. 

2. Operational limits of the simulated scene generator are the amplitudes of the output signal at 
which the signal-to-noise ratio reaches prescribed values, or at which some other factor renders the picture 
unsatisfactory, whichever occurs first. Such other factors might include the ability to interpose cloud 
effects near the edges of model -boards to render the edges unobtrusive, limited display area on an Earth/sky 
projector, or anomalistic picture details in a CGI data base. 

Any convenient linear or angular measures can be used; the measures are referred to a system of Earth- 
fixed axes with the height normal to a nominal spheroid representing the Earth. In the case of model sys- 
tems, the values may be quoted in model dimensions (preferred) or alternatively converted to equivalent 
real-world values, in which case the scale of the model must be quoted. 

The excursion limits can vary from zero to effectively unlimited. The limits in the translational axes 
of an Earth/ sky projector are generally zero; that is, the projector can show no translational movement. 

The displacement limits in the rotational axes are generally unlimited, though maximum velocity and accelera- 
tion are constrained. Velocity and acceleration may be unlimited in CGI systems and, given sufficient data 
bases, linear displacement may be as well. A CGI system will normally respond precisely to the cotnnanded 
input except for a usually fixed transport delay resulting from normal image processing time. However, 
because of the need to pull down a new data base from bulk store, there is a limiting velocity associated 
with an extended translation or rotation. Under certain high-rate conditions, a limit will be reached due 
to the inability to update the on-line data base fast enough. When such a limit is reached because of high 
rotational angles or rates or very high-speed flight, the result is a catastrophic failure of the scene 
generation. 

4.2. 1.2 Measurement methods ~ System limits may be obtained from basic design parameters or by increas- 
ing the magnitude of the command signal until maximum values of the output parameters (displacement, velocity, 
acceleration) are observed. 

Operational limits are measured by applying a sinusoidal input at a number of frequencies for a range 
of amplitudes at each frequency. Each axis is addressed in turn, and the amplitude is increased from near 
zero up to the system limit for that axis; both an upper and a lower limit can be obtained. 

A measurement is made of signal-to-noise ratio, r^ (see Sec. 4. 2. 3. 2) at each measuring point from 
which an interpolation for any specified value of noise ratio can be made. At a given frequency, for 
example, r,^ may be plotted as: 



where a* is the output velocity amplitude given by 




and is the rms amplitude of the fundamental at frequency 



45 


The system and operational limits may be displayed on a log-log plot of velocity vs frequency; thus: 



In the case of CGI systems, a similar technique may be used with the output measured after the digital- 
to-analog converter. However, it is more likely that some other factor may intervene (as may also occur 
with servomechanical systems) in which case its occurrence forms one of the boundaries of the operational 
limits. In these conditions, a tabulation of minimum and maximum values of the parameters for each axis 
may be more convenient than the log-log plot. 

Measurements of positional limits in CGI systems may be made by commanding various locations around 
the extremes of the data base and observing the resulting locations from data base references. Velocity 
limit measurements may be conducted by commanding increasing linear velocity in a selected set of directions 
and observing the point at which the operational velocity limit is reached. Rotational velocity limits may 
be determined by commanding increasing angular velocity for a selected set of orientations and observing the 
point at which the operational rotational velocity limit is reached. For some of these measurements, there 
may be no actual limit to the velocity attainable; however, there will be a practical limit in terms of going 
beyond apparent continuous motion because of the discrete nature of computer image generation. 

4.2. 1.3 Importance of excursion limits — As noted in the preceding sections, excursion limits encompass 
not only displacement, but also velocity and acceleration. When the visual system is being used by the pilot, 
most of the feedback information for control of the aircraft is obtained from motion of the observed scene. 

It is, therefore, vital that it provide a one-to-one relationship with reality. The excursion limits are 
necessary (although not sufficient) criteria for defining the maximum aircraft maneuvers that can be attempted 
while using the visual system. 

Large excursions are likely to be required for simulating high-performance aircraft performing Earth- 
related tasks. (Air refueling, formation flying, and similar tasks that are not Earth-related may be less 
demanding, particularly in regard to displacement needs.) Such aircraft will not only consume the greatest 
volume of space in a short time, with particular implications for model-board systems, but are likely also 
to be capable of generating the highest velocities and accelerations about the rotational axes. 

For all classes of aircraft, the minimum height limit has a special significance in nap-of-the-Earth 
and approach and landing tasks. Apart from mechanical interference between the probe and the model in 
model -board systems, lack of focus and small depth of field may define the limit on the basis of acceptable 
picture quality. 

In CGI systems, insufficient scene content is a possible limit. Exceeding these limits in a CGI system 
causes a degradation of image quality which may result in scintillation-like effects but can also manifest 
itself as a breakup of the image for a few frames. The character and point of such an operational limit is 
largely determined by the formatting of the data base and the updating scheme for on-line data base. 

4.2.2 Time lags 

4.2.2. 1 Definition — One of the important characteristics of visual simulation systems is their non- 
instantaneous response to an input signal. In mechanical visual systems, a camera is typically driven over 
a model -board by a servomechanism. The inertia of the moving parts results in a finite time before the 
camera reaches its conmanded position. For computer-generated imagery, the finite time required by the 
computer to produce a new picture also results .in a noninstantaneous response. The time lag in this case 
is mostly related to the frame time of the computer. 

A convenient measure of the dynamic response characteristics of a system is the describing function. 

The describing function relates the input and output at various frequencies as an amplitude ratio and a 
phase relationship: The values of the describing function are not identical to the transfer function, 
which is strictly only valid for linear systems. Most visual systems will have nonlinear characteristics, 
but where such nonlinearity is small, the describing 'function will approximately correspond to the transfer 
function at the input amplitude and can be considered a linearized representation of the system at the mea- 
surement amplitude. 

Any convenient units may be used in the measurement. However, the amplitude ratio is plotted on a 
logarithm scale and the phase angle on a linear scale in degrees. Amplitude ratio may decrease from unity 
at very low frequency to very small values, tending to zero, as frequency increases. It may exceed unity 
near a resonance in an underdamped system. Phase lag will normally Increase steadily from near zero at 
very low frequency to high values of lag at high frequency. Compensating techniques may be employed to 
improve the phase relationship. 
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4 -2. 2, 2 Measurement methods — The describing function at a given frequency is defined as the complex 
ratio of the discrete Fourier transform (OFT) coefficients for the fundamental frequency of the measured 
output and input signals, which is 


H(k^0 


Xo(ki) 

xTTl^ 


for a sinusoidal input signal. 


For servomechanical systems, an input amplitude of 10% of system limits should prove appropriate. 
However, at the lowest frequencies and where displacement is limited, excessive excursions may result before 
10% of a velocity or acceleration limit is obtained. In this unlikely event (the amplitude ratios and phase 
should by then be unity and zero, respectively), a lesser, and identified, amplitude may be used. 


The results are displayed in the form of Bode plots showing the modulus |H(w)| and phase angle 2^H(u) 
against frequency. A similar technique may be used for CGI systems, but such systems are generally charac- 
terized by more or less fixed-time intervals between the insertion of the input signal and the output signal 
to the video link. This time interval can be measured directly with the output signal accessed immediately 
following digital-to-analog conversion, and, if required, phase lag at all frequencies can be calculated. 


4.2. 2. 3 Importance of time lags and delays - Ideally, a visual simulation system should represent 
the correct spatial position and orientation of the aircraft at all times. This requirement implies a need 
for perfect dynamic response. Such imperfections as occur will either be noticed by the pilot, and so be 
a source of annoyance, or may adversely affect the closed-loop stability of the pilot/airframe/display 
control loop. In the latter case, the pilot will employ a control strategy in the simulator that is dif- 
ferent from the one used in flight, with a consequent reduction in the validity of the simulation. 


Visual simulation systems undoubtedly suffer from dynamic response deficiencies. In mechanical visual 
systems, a camera is typically driven over a model -board by a servomechanism. The inertia of the moving 
parts results in a finite time before the camera reaches its commanded position. For computer-generated 
imagery, the finite time required by the computer to produce a picture also results in a noninstantaneous 
response. The time delay in this case is mostly related to the frame time of the computer (see Sec. 4.2.7). 
Such lags and delays are additive to dynamic deficiencies from other sources and any analysis of the 
dynamic response of the simulator must include all sources. 

No universal set of requirements for the dynamic performance of a visual simulation system can be 
postulated; the particular eopli cation of the visual system is all-important. Three factors must be 
considered: 

1. Type of aircraft: whether the vehicle itself has the capability of fast response, and whether it 

is inherently stable 

2. The mode of flight: the tasks to be performed and the precision of control that is required 

3. The degree of freedom under consideration: requirements on rotational response are usually more 

severe than those on translational response 


These factors are interdependent. For example, the most demanding requirement on rotational response for 
a military combat aircraft will be in the rolling plane (up to 200°/sec). Conversely, the maximum achievable 
yaw rate may be the design case for the simulation of a helicopter in low-speed flight or hover. 

It is self-evident that the visual simulation system should be able to match the aircraft in dynamic 
response. Less obvious is the importance of the aircraft's inherent stability (or lack of it). In most 
modes of flight, modern aircraft have good stability about all axes although, in some cases, this is pro- 
vided by a control and stability augmentation system. The pilot task is then simplified to one of kinematic 
flightpath control, in a quasi-open loop fashion. If the pilot is required to stabilize the vehicle (e.g., 
after a control system failure, or in hovering flight), the requirements on the dynamic response of the 
visual simulation system are more severe, since the pilot's ability to achieve closed-loop control is 
degraded by the presence of lags or time-delays in this system. In such circumstances, a 0. 2-sec lag in 
the display of bank angle will result in degradation of control (Ref. 54). A 0.4-sec lag in height informa- 
tion affects controllability of height in the hover (Ref. 55). 

There are other examples that illustrate the importance of display lags and computing time delays. For 
instance, TV model -board visual systems have not given sufficient realism near the ground to represent the 
landing flare. On the other hand, the CGI systems now seem adequate for airline training, including the 
flare. One important change is the elimination of the camera-drive mechanism, and the associated imperfec- 
tions, particularly in height response, which become critical for height control near the ground. 

Air-to-air combat simulation is a particularly demanding case with respect to dynamic response; very 
high rates of maneuver are used, high positional accuracy of images is required, and the computational load 
is high. Present-day combat simulators have difficulty in reproducing fine aiming tasks, such as gun track- 
ing. Significant contributions to this deficiency come from the computer time delays and the response 
characteristics of the image projectors. 


4.2.3 Noise 


4.2.3. 1 Definition — Noise Is considered here to be the perturbation of the output displacement signal 
from its nominal value. This nominal value has the same frequency as the input sinusoid with amplitude and . 
phase obtained by minimizing the mean square of the difference between the nominal value and actual output. 
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In servomechanical systems, operation in one axis may induce movement in one or more of the others. 
Consequently, two kinds of noise are distinguished: 

1. Noise in the stimulated degree of freedom 

2. Parasitic noise in the nonstimulated degrees of freedom 


The same units are used as for excursion limits. In addition, nondimensional values are determined, where 
the standard deviation and peak noise are quoted as ratios of the fundamental of the output. 


The design aim is always to achieve minimum noise. In general, the absolute value of noise increases 
as the input amplitude is increased. On the other hand, the noise tends to some finite amplitude as input 
amplitude is decreased to zero. 

4. 2. 3. 2 Measurement methods - Noise is computed from the time histories recorded for the excursion 
limit measurements. Analyzed results are presented at only one or two frequencies (say 0.1 Hz and 0.5 Hz) 
over amplitudes from zero up to the excursion limits. 


The spectral power distribution is obtained from the OFT of the sampled signal. The variance of the 
following components is then computed: 


Fundamental output = a^Ck^) ; say k^ = 1 and 5 (freq = Hz) 


m 

Acceleration noise ^ o^(k-j) - 

i*i 

Elimination from the OFT of X(k) for k *= k£ and k > m (where m is normally chosen as 35 Hz) and use of 
the inverse Fourier transform gives a reconstruction of the noise signal, (See Appendix C.) From this the 
peak value, Ap, of the noise is determined. 

The following dimensionless ratios are immediately available: 

Modulus of the describing function lH(w, A)] = Of/a^ 

Noise ratio r^ » a^/of 

Peak ratio = Ap/(of • v?) 

For the undriven axes, the standard deviation and peak ratio can be calculated directly from the recorded 
time histories. 

The results are presented in a series of plots that might look something like the following: 



xPEAK 
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In practical simulation, the linear translations are often driven at more or less constant velocity, 
rather than sinusoidally. Consequently, for these axes a constant velocity input signal, from zero up to 
the maximum achievable, should also be used. The mean output velocity is then subtracted from the output 
velocity time history (or digitally differentiated output position) and the standard deviation and peak 
value of the velocity noise can be calculated and presented as above. 

4. 2. 3. 3 Importance of noise - Noise in the output displacement signal causes a reduction in resolu- 
tion. In addition, if the noise is of such a magnitude as to be perceptible (greater than about 2 mrad rms 
as viewed on a normal TV display), it will be distracting and irritating, and in an extreme case of high 
amplitude at relatively low frequency (less than, say, 1 Hz), it may induce control responses in certain 
tasks (e.g., weapon aiming). 

4.2.4 Linearity 

4.2.4. 1 Definition - Linearity is the degree of proportionality between the output displacement and 
a command input displacement when the latter is unidirectional. Comparison of the output signal with a 
steadily increasing input signal is a simple means of determining linearity. However, it is also useful 
to consider the distortion (harmonically related) of the output from a sinusoidal input, particularly of 
the second and third harmonics, as an indicator of low-frequency nonlinearity. The information is readily 
extracted from the measurements made for excursion limits and allows the components of noise attributable 
to nonlinearity to be distinguished from that at higher frequency (due to higher harmonics and stochastic 
sources). 

The linearity is quoted as a ratio, either as the deviation from a proportion of the input conmand, or 
as the ratio of the fundamental output to a sinusoidal command. Feedback elements of high precision are 
available, and nonlinearity should be very small. Nonlinearity is far more likely to be prominent in the 
video link, as a function of location on the display screen, than in the scene generator. 

4. 2. 4. 2 Measurement methods - The two alternative methods are: 

1. After an oscillation to eliminate hysteresis (backlash), the input signal is Incremented and the 
difference between the output and the command is plotted as a ratio of the command. 

2. From the recordings made for determining excursion limits, the variance of the following component 
is calculated. 

For the fundamental output of^ « a^(kjj^), the low-frequency nonlinearity is o|n = a^(k£) + o^(3k,*); 
where ko ~ 1 and 5 from which the low-frequency nonlinearity ratio r-n * <J£n/®f modulus 

|H((j, A)| are calculated and plotted against amplitude. 


A 



A 

4. 2. 4. 3 Importance of linearity - Lack of linearity in the scene generator is unlikely to be perceptible 
to the pilot. Nonlinearity In the video link causes two effects: 

1. A feature moving across the viewing screen at nominally constant velocity will appear to be accel- 
erating or decelerating. 

2. The perspective of features will be statically incorrect and will change inappropriately as they 
move across the screen, leading to a false judgement of position. 

Quite small departures from linearity can lead to large errors in the computation of miss distance following 
weapon delivery. Unless the system is suitably zeroed, similar errors can occur, for example, in the mea- 
surement of touchdown performance. 

4.2.5 Hysteresis 

4.2. 5.1 Definition - Hysteresis is the difference in output displacement resulting from the same 
magnitude of command displacement in opposite directions of movement. Contributions to hysteresis may come 
from many sources, of which the most comrion are friction and backlash in the drive system or feedback elements. 

Any convenient linear or angular measures may be used to refer to Earth-fixed axes. Visual simulation 
system dimensions, rather than equivalent real-world values, are preferred. Hysteresis will normally be 
very small in system dimensions. However, when scaled to real-world equivalent on servomechanical systems, 
it may give rise to substantial errors - tens of meters in location and up to a degree or more of error in 
rotations. 


4.2. 5. 2 Measurement methods — The most convenient method is an input of a very low-frequency sine 
wave, low enough to avoid the effects of dynamics of the system. On freedoms with limited excursions, 
the amplitude should be as large as possible without incurring any limiting factor. For unlimited freedoms, 
the amplitude should be to the limits of repetition; for example, 360° in rotation. Sometimes hysteresis is 
different for very small conmand amplitudes; if so, this condition should be evaluated by additional 
measurements. 
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OUTPUT 

DISPLACEMENT 

ERROR 



It is generally more informative to plot the 
results as output displacement error , rather than 
displacement, against conmand displacement, as 
shown in Figure 52. 

4.2. 5.3 Importance of hysteresis - Hysteresis 
has the effect of creating a difference in the com- 
puted and observed position of the aircraft. In 
many respects, it causes similar errors to those 
produced by nonlinearity; for example, incorrect 
assessment of miss distance. However, at small ampli- 
tude where the conmand input is of the same order of 
magnitude as the hysteresis loop (output error), 
instability or limit cycling of the pi lot/aircraft 
combination will occur. Low hysteresis is thus clearly 
important where gentle maneuvers of high precision are 
required, such as air-to-air refueling or precise 
hovering (but see also "threshold"). The final phase 
of the flare prior to touchdown is often a condition 
where all variables (except the longitudinal displace- 
ment) approach zero and many reverse direction, and 
where hysteresis may seriously affect performance. 


4.2.6 Threshold 

4.2.6. 1 Definition — Threshold is characterized by the need for the input signal to be at a certain 
minimum level before any output is obtained. However, in the case of velocity- driven systems, the ever- 
increasing position error following the smallest input means that ultimately a response will be obtained. 

In this case, the time to respond is significant, and may be considered the dynamic threshold. 

There will always be some threshold although it is possible that it may be of such a magnitude as to 
be masked by noise. 

4. 2.6. 2 Measurement methods — A step input signal either of position (for position-controlled systems) 
or of velocity (for velocity-controlled systems) is increased in magnitude from zero until a system output 
is detected (the position loop of velocity-controlled systems is open, i.e., feedback disconnected). 

Alternatively, in the case of velocity-controlled systems with a position-feedback drive, several step 
inputs of velocity, increasing from near zero to some reasonable value (operational limits where appropriate) 
are injected. The time for the output position to reach 63% of the input position is plotted against the 
command input. 



4. 2, 6. 3 Importance of threshold — By definition, threshold is significant where a small change from a 
zero input condition is required. The size of the small change is particularly important in the hovering 
vehicle, but is also important where any one of the degrees of freedom is to be held nominally zero, for 
example, in level or rectilinear flight, and particularly close to the ground. As in the case of hysteresis, 
high threshold may lead to instability or limit cycling. 

4.2.7 Further remarks on CGI systems 

The dynamic characteristics of the scene generator of CGI systems are often defined by a limited number 
of parameters, which can readily be measured, and from which the equivalent values of the foregoing 
(Secs. 4.2.1 through 4.2.6) characteristics can be readily deduced. These parameters are: 

1. Transport lag: The time between the injection of a command input signal into the visual system 
computer and the output from the digital-to-analog converter driving the video system. The technique for 
measuring the transport lag is to input the visual system computer to point the visual system toward the 
blue sky. Then a red hue is inserted in the visual system's color register for the ground. The input to 
the computer corresponds to changing the pitch angle by 90® in one iteration. The image generator-to-di splay 
time or transport lag begins with the input to the visual system computer and ends when the voltage on the 
blue gun goes to zero and the voltage on the red gun increases. The maximum total time lag is equal to the 
transport lag plus the generator time to update one frame plus the display trace time to refresh one frame. 
For example. General Electric COMPU-SCENE “1000," with its 60-Hz update and refresh rates, should have an 
image generator-to-di splay time of 50 msec, to which should be added 16.67 msec for first field time loss 
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plus 16.67 msec for display trace time for a total of 83.34 msec. For the COMPU-SCENE *7000," with its 
30-Hz update rate and 60-Hz refresh rate, these figures are 100 msec, 33.33 msec, and 16.67 msec, so that 
the maximum time lag is 150 msec. 

2, Word length: The number of bits of information necessary to describe a scene element and to fix 

its location. 

3, Scaling: The magnitude represented by the least significant bit used in controlling a parameter. 

4, Update rate: The frequency at which the coordinates of any element in the generated scene are 

updated (frames® generated per second). 

4.3 Video Link 


4.3.1 Refresh rate 


To create the illusion of smooth continuous motion, television-based visual systems must create and 
display a new picture at a sufficiently high rate. Motion obtained in this manner is called apparent motion; 
it is discussed in Section 4.5. One of the unfortunate effects of producing moving pictures in this manner 
is the creation of a perceptible modulation of the picture luminance if the rate is below a certain frequency. 
Television engineers found they could eliminate this modulation or flicker by using an interlace technique 
in which successive pictures were composed of either the odd or the even-numbered raster lines. Each of these 
interlaced pictures is called a "field" and two successive fields are called a frame. The advantage of this 
technique is that a picture with a given resolution can be displayed with no flicker, using only half the 
video bandwidth of a noninterlaced system. Higher-order interlaces are possible but rarely used. Refresh 
rate is the same as field rate in raster scan systems, except for field sequential color television, in 
which it should probably be used to describe a complete color sequence. In calligraphic systems it is used 
to describe the number of complete pictures drawn each second. The refresh rate at which flicker is just 
imperceptible is called the "flicker suppression refresh rate." 

The appearance of visually perceptible flicker in television displays depends on the refresh rate, the 
brightness, color, and size of the illuminated area and, to some extent, on the decay characteristics of the 
display. It also depends on the level of ambient luminance and the position of the image on the retina. Two 
general categories of flicker are spoken of in display literature - small -field flicker and large-field 
flicker. Small-field flicker usually refers to the flicker of elements in single lines or small groups of 
lines having a visual angle that does not exceed that of the foveal vision, that is, a solid angle of about 
3°. Small -field flicker restricted to two or three lines is sometimes referred to as interline flicker. 
Large-field flicker refers to flicker appearing in substantially all portions, of the face of the display. 
Large-area flicker appears as random movements across the display created by some line dot interlace patterns 
collected in groups; it appears much more objectionable than the small, single-line, single-element flicker 
or scintillation. This snowflake effect can be greatly reduced by the use of long-persistence phosphors. 


The frequency needed to suppress flicker is 
strongly dependent on phosphor persistence. In 
Figure 53, the relationship between CRT refresh rate 
for perceptible small -field flicker as a function of 
the average luminance of the spot above the surround 
for a variety of phosphors is shown. Seven phosphors 
were tested (Ref. 56); they had a persistence range 
from 38 ysec to 550 msec. They were, in order of 
their increasing persistence, P-1, P-4, P-7, P-12, 
P-20, P-28, and P-31. 

Differing persistences of the phosphors are not 
the sole cause of the differences in refresh rates 
required to eliminate flicker. Phosphor P-28, for 
instance, is a long-persistence phosphor and at 
high brightness levels its required refresh rate is 
above that for a P-1 phosphor, which is a medium, 
short-persistence phosphor. It is suggested that 
the high primary flash of the P-28 may account for 
its unusual production of flicker. The primary flash 
is a term sometimes used to describe the very bright 
fluorescence of a cathode luminescent phosphor when 
it is being bombarded by the electron beam of the 
CRT. 

' In interpreting these results, it must be kept 
in mind that the long-persistent phosphors are not 
suitable for displaying moving images. The persis- 
tence causes the images to smear. 



ABOVE SURROUND, cd/m^ 

•Fig. 53 Flicker suppression refresh rates for a 
small area and several phosphor types 


The large-area flicker suppression rate fora two-to-one interlaced system using a P-1 phosphor is 
about 42 Hz when the luminance is about 20 nits. At luminances approaching 100 nits or if the field of view 
extends into the far peripheral areas of the retina this value will increase to 50 or 60 Hz. 


One other effect which should be mentioned here is line crawl. If an observer stares at a uniformly 
colored area on a 2:1 interlaced TV display he usually reports seeing a set of lines, parallel to the raster, 
slowly moving up the face of the display. The effect is a manifestation of apparent motion in which the 


®A frame is one complete scan of the image area by the electron beam. A frame consists of two (or 
more) interlaced fields, or all of the visible scan lines. Also called a raster. 
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human visual system interprets the two successive interlaced fields as one field moving up the screen at a 
rate equivalent to the raster spacing each field- time. Another type of motion may occur which is limited 
to small fields. Short segments of an individual line in the picture may appear to move back and forth 
between two adjacent scan-lines in an oscillatory motion. 

Apparent motion is often referred to in the psychophysical literature on vision as stroboscopic motion 
or as the phi phenomenon (see Sec. 4.5). The appearance of this phenomenon is strongly Influenced by inter- 
lace techniques. The effect of sequential and staggered interlace on line-crawl indicates that all higher- 
order sequential line interlaced rasters have unacceptable line-crawl characteristics. Certain staggered 
sequences have been found better than others particularly when used with the long-persistence phosphors. 


4.3.2 Update rats 


The update rate is the generation frame rate or the frequency at which a complete image is generated. 
This is in contrast to the refresh rate discussed in the previous section. A new picture cannot be produced 
at a rate greater than the update rate, and, therefore, there is an additional time lag (over and above the 
transport lag defined in Sec. 4.2.7). If a line-scan, digital -computer generated system has refresh and 
update rates of 60 Hz, the effects discussed below are minimized. However, if the update rate is. say, 

30 Hz, certain phenomena will be observed. If the scene being drawn contains lines that are parallel to 
the flightpath of the airplane, these lines will appear to be quite smooth as long as the airplane is pro- 
ceeding on a straight course paralleling these lines in the scene. However, if the airplane begins to 
change heading, it will be observed that when the angular velocity exceeds a certain amount, these lines 
will become jagged instead of smooth. It is as though the image were made up of two combs with every other 
tooth being represented by a different comb. If one comb is moved to the left and the other to the right, 
then the tips of the teeth will no longer form a straight line. Figure 54 is an illustration of how it 
would appear, in a normal scene on the left and in the rapidly turning scene on the right. This is an inter- 
action between the interlace, the update rate, and the angular velocity generated by the motion of the air- 
craft relative to the scene. 




60 Hz UPDATE RATE 


30 Hz UPDATE RATE 


Fig. 54 Effect of high turning rates and two update 
rates on the appearance of high-contrast 
runway edge and center lines 


A different anomaly occurs when the aircraft 
remains on the same heading and the pitch is changed. 
When the rate of change in pitch becomes sufficient, 
the steady transition changes to a step-wise progres- 
sion. This is most noticeable when looking at hori- 
zontal elements. The vertical lines are unaffected 
by this direction of movement. The interaction between 
the update rate and the depiction of small objects in 
the scene is represented by a phenomenal change. 

First, the width of the small object increases, then 
it appears as two elenents instead of one. An example 
of this is when a scene is a runway and the dynamic 
movements are those of a takeoff. As the aircraft 
gains speed, the runway edge lights will at some point 
begin to stretch and then break apart and become two 
lights. The space between the two representations 
will increase as a function of the angular velocity. 

For example, with a 1000-line CGI 60-Hz raster display 
at a 30-Hz update rate, with a 9-m eye height, and an 
aircraft speed of about 116 knots, the perception will 
be that of double images on runway lights 150 ft to the 
right or left of the cockpit, but still within the 
forward display. The double images are located in the 
lower right and left portions of a 30° x 40° field of 
view. Pilots may, incidentally, learn that a relation- 
ship exists between the speed and the appearance of 


doubling of the lights. Without conscious effort, they could use this as a cue for the aircraft rotation, 
possibly without any reference to the airspeed indicator. The doubling of the runway edge lights will also 
occur during the taxiing maneuver, when the aircraft turns from the runway onto an adjacent taxiway. This 
is true, for example, for the same display as above, for a 90° turn at a taxiing velocity of about 9 knots. 
Pilots learn to associate the amount of separation of the images of a single light with the angular velocity 
of a turn and use it to their advantage in the simulator. 


For given update and refresh rates, the perception of edge breakup and doubling of small objects is 
directly related to the angular velocity in the scene. High aircraft speeds, rates of turn, roll rates, 
and so forth may occur without the pilot seeing this visual phenomenon. For example, for the same aircraft 
speeds, near-objects in the periphery may appear to double, intermediately distant targets will appear to 
blur, and distant objects will appear unaltered, as shown in Table 4. The advantages of using a faster 
update rate are that the image quality is noticeably better and the simulator and its visual scene can be 
used for more tasks; however, the disadvantage is that the amount of computer capability will have to be 
increased accordingly. 


4.3.3 Eye movement 

A visual anomaly occurs when the simulated aircraft is stationary and the pilot is scanning among the 
various displays available to him. The head and eye movements will interact with the refresh rate in this 
situation. When he changes his gaze from one position to another, he may notice (during the saccade) an 
interaction between the writing rate on the raster lines and his eye movement. A similar visual anomaly 
occurs when the simulated eye reference point of the pilot is moving in space and he in turn is looking 
among the different displays. Since both the raster lines and the elements in those raster lines are gen- 
erated sequentially, these two factors may interact with eye movements. When the eye is in motion, the 
human visual system will tend to suppress the perception of elements in the scene, reinstating the perception 
again at each pause; so there may be an interaction between the timing sequences of the eye movements and 
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TABLE 4: The alteration of the appearance of small objects (|3 arcmin) seen in the 

periphery of the eye as a function of distance and velocity in a 1000-line 
CGI 60-Hz raster display with a 30-Hz update rate 


PARADIGM 



*WILL APPEAR AS TWO OBJECTS INSTEAD OP ONE. 
••WIDTH OF OBJECT WILL APPEAR TOO LARGE. 
•••OBJECT WILL APPEAR UNALTERED BY SPEED. 

(VALUES IN BODY OF THE TABLE ARE IN DEGREES AND 
REPRESENT THE ANGLE TRAVERSED IN 33 milliseconds) 
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the writing rates that are occurring during saccades (eye movements). The resulting perception in which the 
image appears to jump Is not the result of too slow an update rate but is, rather, an Interaction between 
the updating and the natural pauses in the visual perception. 


4.4 Suggested Methods of Measuring the Complete Visual Simulating System 


. determining the temporal properties of a simulator visual display is the impracti- 

cability of making all measurements at the eyepoint. Moreover, some of these parameters may, in practice, 
be controlled by the host (i.e., the aircraft) computer in a given installation. Since it is the capabilities 
of the visual simulation system alone (rather than its performance in a particular installation) in which we 
are interested, it is preferable to identify these parameters, wherever possible, with the maximum achievement 
of the visual simulation system. 


If it were possible to analyze cine photographs taken from the eyepoint, many of the problems of mea- 

Initial synchronizing mark on the film, a film reading 
system might be developed to define and digitize the location of an object in the scene. (The cine frame 
rate would have to be about 5 times the frame rate of the TV type of display or would have to be at least 
^ice the highest frequency of Interest of a continuous type of display.) Given that cine filming is imprac- 
ti cable, a popible alternative utilizing photodetectors covering two scan lines (assuming a 2:1 interlace), 

and attached to the face of the display unit, has been suggested as a means of determining the dynamics of 
the complete video link. ^ jriami 


For shadowgraph systems, an array of photodetectors set in the display surface, and illuminated by a 
sharp line frm the shadowgraph model, can provide an electrical output proportional to displacement, from 
which a direct comparison between input and output can be made. A single row at different orientations 


" A similar technique for the measurement of TV systans would appear to present considerable practical 
problems, if it were feasible to arrange for a strip of adjacent photodetectors, each one encompassing two 
(and only two) raster lines, then it would be possible to take 50/60 samples of locations of a bright line 
per second, that is, at field rate. Fiber optic light guides are a means of accomplishing such an array. 

A practical maximum frequency for a sine wave input would then be about 5 Hz. The pulses from the detectors 
would need to be converted, either to an analog signal of magnitude proportional to the location of the 
detector and then via an analog-to-digital converter to- the digita'l computer, or each combined directly 
into a digital word for transmission to the computer. Analysis would follow identically the procedures of 
Section Ideally, the array should be oriented both vertically and horizontally to cover the various 

degrees of freedom and, simultaneously, to measure picture linearity, etc. A less desirable alternative 
is to reorient the viewpoint in roll or yaw so that one direction of the array will cover all axes. 


Another possible technique, applicable only to horizontal movement across the display, is to select an 
individual raster line and measure the time interval of the bright-up pulse from the beginning of the line 
In this case, only 25/30 samples per second are possible and the characteristics of the display are excluded 
For simple analysis via a direct input to a digital computer, it is necessary that the computer user have 
access to a megahertz clock, which is not normally the case. Reorientation of the viewpoint can again cover 
all axes. 


For these TV sys^s, amplitude variation of the servo drives (say, 90% of picture size) is accomplished 
by varying the range (on the model) of the viewed feature - near the entrance pupil for small amplitude of 
translations and large amplitude of rotations, and far away for large amplitude of translations and small 
amplitude of rotations. 
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4.5 Interactions for the Complete Visual Simulator System 

4.5.1 Introduction 

In Section 4.1 the "video link" was defined as including all elements of the visual system associated 
with the visual pathway. In a camera/model -board system, the start of this pathway is at the entrance pupil 
of the optical probe (though it could be argued, pedantically, that any "shimmering" of the picture caused 
by changes in the refractive index of the air between the model-board and the lights due to heating was also 
a property of the video link) and the finish of the pathway is at the pilot's eye, which practically means 
on his side of any collimating optics. For a CGI system, the finish is the same, but the starting point is 
less clear. However, it can be argued that only after digital-to-analog conversion is complete is a signal 
available from which a picture can be generated. This point has been taken as the end point for the defini- 
tion of transport lag (Sec. 4.2.7), a characteristic of the scene generator; consequently, it is a logical 
starting point for the video link. 

In the discussion of Sec. 4.3, we addressed certain temporal properties specifically related to this 
video link alone. However, the discrete nature of the scene created by the scene generator in CGI systems 
causes additional effects on the viewed picture when considered in conjunction with the video link. These 
"interactions" also are discussed. 

4.5.2 Dynamic interactions with visual mechanisms — illusions 

Whenever an image is formed or transmitted by a sampling process such as a television raster, inter- 
actions between the sampling frequency and similar spatial frequencies in the image cause the creation of 
alias frequencies. Sampling theory (from Nyquist's theorem) states that any spatial frequency in a sampled 
image greater than half the sampling frequency will generate alias frequencies. A rigorous analysis of 
image reproduction by a television raster was given by Schade (Ref. 23). He recommends prefiltering of the 
image to eliminate all frequencies greater than half the sampling frequency. In a conventional model -board 
visual system, the MTF of the optical probe and the spread function of the scanning beam in the TV camera 
will attenuate the higher spatial frequencies sufficiently to reduce aliasing to acceptable levels. Certain 
periodic images with line structures essentially parallel to the raster will, however, generate alias fre- 
quencies. No aliasing will occur with spatial frequencies perpendicular to the raster. 

CGI systems that create images on a pixel -by-pixel basis are subject to the more serious problem of 
aliasing caused by two-dimensional sampling (Ref. 23). A rigorous spatial prefiltering computational process 
to limit the image to sub-Nyquist frequencies (i.e., less than half the sampling frequency) would be extremely 
time-consuming. All CGI systems use some form of image smoothing or antialiasing algorithms in an attempt to 
reduce aliasing effects to acceptable levels. Gardner and Berlin (Ref. 57) and Schumacker (Ref. 58) describe 
methods for prefiltering of the image which appear to be effective. It should be noted that postfiltering 
of the image by defocusing the displayed image cannot eliminate the low-frequency aliasing caused by the 
image computation. It is, however, possible to reduce aliasing caused by the raster structure of the shadow- 
mask of a color CRT by defocusing the electron beam or by the use of vertical "spot wobbling" techniques. 

Dynamic interactions would be the more general term for the problem called aliasing and it includes a 
multitude of effects which visual specialists classify as visual anomalies. The basic underlying physical 
explanation is that they are due to quantizing and sampling at various stages of the generation, processing, 
and display of raster- type images. Flight instructors and pilots undergoing training have some very descrip- 
tive names for these visual anomalies: shearing, tearing, flickering, creeping, sparkling, streaking, bounc- 

ing, scintillating, racing, jumping, skipping, edge walking, and reversing are examples of such descriptors. 
The potential effects of these visual phenomena have not been quantified. They are at least annoying and 
create distractions that may interfere with pilot performance by causing frequent short- interval delays 
thereby increasing the pilot's workload. In addition to being distracting, they can impose incorrect per- 
ceptions of speed when they take the form of slowdowns or movement reversals. They may provide navigational 
and spatial orientation cues that do not exist in the real World and thereby lead to negative transfer of 
training. 

An example of a potential source of negative transfer of training occurs in the computer- genera ted image 
wherein the runway edge lines are represented by narrow white lines on the black macadam surface. At some 
distance from the field, these extended lines are represented by a small number of raster lines. The inter- 
action between the digital writing of scan-lines and elements generates an apparent movement of these runway 
edge lines. In a left turn to final alignment with the center line of the runway, the pilot may observe 
that the runway edge line closest to him appears to be moving from the horizon toward him while the one on 
the far side of the runway is apparently moving away from him toward the horizon. If he is observant, he 
will notice as he lines up with the center of the runway that these apparent motions slow down or become 
stationary and if he overshoots to the right of the runway the apparent motion reverses and speeds up. The 
bright pilot soon recognizes this as a cue of alignment and he can use it effectively in the simulator. How- 
ever, this new skill has no transfer to the real world, for such perceptions only exist in the digital line- 
scan type of imagery. Just as the above cue helps him with the azimuth alignment with the runway, there is 
another that provides equivalent help in altitude. If, for example, there is a relatively narrow object 
beyond the end of the runway, it may appear on only one or two raster lines. In this case, the pilot may 
find that, as he descends too low, the object, disappears. If he is on or above the glide slope, the object 
will be in view. Therefore, he may maintain a very good glide-slope angle if he just keeps that object in 
view; in effect, he has his own VASI. Software introduction of a cutoff angle may be used to eliminate 
small objects from becoming just visible at glide-slope angles. However, if such software correction does 
not exist, the pilot has an aid in the simulator that does not transfer to the real world. 

Before the advent of smoothing techniques (1973), surfaces with sloping edges appeared as a series of 
steps or jaggies. Each pixel was assigned supra threshold luminous chromaticity that was based on a single 
sample of the scene spatially located at the center of the pixel. If the horizontal length of the pixel was 
greater than the resolution threshold of the eye, then the edge was discernible as a step that, in instances 
of changes in roll, made lines like the horizon appear to shear or move from the left to the right. This 
apparent motion is a special case of what is known as the phi phenomenon, or the relative positional change 
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due to a temporal sequence in the absence of a real-object positional change. An effective solution to 
such problems in computer-generated images is the technique of smoothing in which the edge gradient in both 
hue and luminosity transition is reduced by displaying each pixel that is cut by an edge as the blend of 
colors on either side of that edge. The future appears to offer formidable techniques in smoothing and these 
may be effective in reducing what we have been calling aliasing. However, as we mentioned earlier, there is 
an interactive effect of smoothing on resolution. The jagged appearance of the edge will be maximized by 
high contrast and decreased by lowering the contrast. 

We turn now to considering the contribution of the display to some specific visual phenomena or anomalies 
such as flicker, shearing, strobing, raster-line dominance, and apparent motion. Some factors that produce 
these anomalies are type of scan, formatting, frame rate, field of view, visual system lag, update rate, and 
refresh rate. In particular, those visual phenomena that affect the sharpness or displayed resolution are 
generally due to interactions between two or more of the following characteristics of the visual simulation 
system: the number of active TV lines per unit of visual angle, the number of picture elements horizontally 

along each of the lines, the uniformity of line resolution, the depth of field in the scene, the phosphor 
decay time, and spot size spread and shape. For those that affect the pilot's reported eyestrain, fatigue, 
or difficulty with size and distance relationships, generally the problems occur due to display collimation 
and image distance error, or the size of the viewing region. Included in these characteristics are the 
lateral vergence between the two eyes or dipvergence as well as the image distance and its variability. Also 
affecting perceptual orientation and strain are the geometry of the perspective, the eye-relief envelope, the 
reflections, glare, ghosting, scratches, seams in the display, the difference in image sizes as displayed to 
right and left eye of the pilot, and differential distortion between the images for each eye. Factors affect- 
ing luminance intensity can be temporal as well as constant and these will also affect display contrast, 
resolution, perception of distance, and eyestrain. Color may affect these as a consequence of temporal changes 
in color balance and chromatic fringing and misregistration. 

The scan mechanism of television-based displays is responsible for many-of the effects broadly described 
as aliasing. One other interesting dynamic effect due to the sequential nature of the scan mechanism is the 
tilting of moving images. 

The image on the face of a conventional television camera is scanned by an electron beam moving sequen- 
tially down the image in a series of horizontal lines. Neglecting the vertical retrace period this scan is 
accomplished in one field time. If the image of a talT vertical object is caused to move across the face 
of a TV camera (see Fig. 55), the resulting video pulse in each horizontal scan will be displaced by an 
amount proportional to the velocity of the image. The resulting image on a display will show a tilted object. 

When this is observed by an eye that tracks the moving image, the resulting image on the retina is not only 

stationary but is also vertical. If, however, the observer fixates on a stationary part of the image, the 

moving vertical object will appear tilted. Integration effects in the camera or display will increase the 

width of the moving object but will not affect the angle of tilt. In a typical current visual simulation 
system, an individual display subtends an angle of about 36° vertically, A vertical object moving across 
this display at 100°/sec would appear to have a tilt of about 3°. 

The opposite effect should be apparent on CGI systems that calculate the image position at each field 
time. The tilt will only appear when the eye is tracking the object, but the maximum tracking rate of the 
eye (without significant degradation in resolution) is only about 20°/sec. The maximum tilt that can be 
observed therefore is about 0.6° on a display with a 36° vertical, field of view. This may be below threshold 
as no report of the effect is known. 
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Fig. 55 Illustration of tilted image phenomena 
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4.5.3 Implications for foveal viewing and peripheral viewing at different luminance levels 

Crozier and Holway in 1939 (Ref. 59) determined detection thresholds along the zero to 180° meridian 
ranging on the retina from 10° on the nasal side to 32° on the temporal. (They thus avoided the blind spot.) 
As indicated in Figure 56 (which is based on the data of Ref. 59), the threshold of photopic luminosity 
for white light is much higher in the fovea than in the periphery. These data imply that if, in the visual 
simulation system, we supply sufficient limiinous intensity for the fovea we can expect to have provided 
sufficient intensity to be above the threshold for the periphery. 

However, the problem is a little more complicated when we consider color. The color fields of the 
eye differ, with the red and green being relatively smaller, the yellow, blue, and white much larger. The 
data shown in Figure 57 are just the comparison between the blue field and the red field from the data of 
Haines (Ref. 60). Visual response time is the time that elapses between the appearance of a stimulus and 
the desired response • It should be noticed in Figure 57 that the indicated isoresponse time zones of the 
blue field are well within those of the red field. The implication here is that to be equally perceived by 
the peripheral portions of the eye, peripheral stimulation that is predominantly in the red end of the spec- 
trum must have higher intensity and contrast than if it were blue. The influence of the relative sensitivity 
to color in the periphery may have an interesting interaction with the difference in the individual's chromo- 
stereopsis so as to influence his perception of distance, orientation in space, and recognition of speed. 
Also, his interpretation of the blue color representing an atmospheric haze may be different from atmo- 
spheric haze represented only by luminous intensity. These possible perceptual influences may dictate the 
relative value of specifying the requirements for chromatic or achrwnatic displays. 



DISTANCE FROM FOVEA, deg 

*troland IS A UNIT OF RETINAL ILLUMINANCE EQUAL TO 
THAT PRODUCED BY VIEWING A SURFACE WHOSE 
LUMINANCE IS 1 candle per square meter THROUGH AN 
ARTIFICAL PUPIL WHOSE AREA IS 1 square millimetar 
CENTERED ON THE NATURAL PUPIU 

Fig. 56 Foveal vs peripheral thresholds for lumi- 
nance Intensity' and wavelengths 


BLUE 

2.50 X KT^ cd/cm2 



RED 

8.56 X 10^ cd/cm2 



Fig. 57 Retinal isoresponse time zones for binocular 
visual field and blue and red stimuli 


4.5.4 Spatial-temporal Interactions 

The human visual system is primarily responsive to dynamic, rather than static, stimuli and the motion 
.detectors respond similarly to discontinuous and continuous motion. This is analogous to the perceptual, 
phenomenon variously called the phi phenomenon, beta apparent motion, and stroboscopic motion. It is the 
appearance of smooth and continuous motion that can be generated by a succession of stationary, displaced 
images. However, as noted by Braddick and Adlard (Ref. 61), apparent motion relates to the interpretive 
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process of perception and not simply to the known neurophysiological mechanisms. The visual phenomenon of 
apparent motion can only occur by virtue of some visual mechanism that detects the spatiotemporal relation- 
ship between elements in successive exposures of stationary, displaced images. 

Some of the effects of the phi phenomenon in visual simulation systems have already been discussed 
in Section 4.5.2. Additional effects are discussed in this section as they relate to spatial -temporal 
interactions. 

C61 manufacturers are introducing systems that use an overlay of texture of a particular, uniform 
geometric pattern to identify a surface. When the viewpoint is moving through space and the pattern viewed 
is written by line- scan imagery, in which the elements of the line scan are digitally introduced, there is 
a temporal interaction. For example, on flying toward a runway during an approach to landing, the larger- 
sized elements of the pattern in the immediate foreground may permit a perceptual interpretation of moving 
smoothly through space. However, at an intermediate distance, the geometric pattern may produce an inter- 
action due to size and writing rates such that this appearance of smooth motion no longer exists. This is 
known as a stroboscopic effect. 

Such a stroboscopic effect was evident in the initial development of the COMPUSCENE data bases prepared 
by 6.E. for Boeing. The FAA specification that the centerline of the runway consist of ‘150- ft- long painted 
segments followed by equal lengths of unpainted intervals, is the perfect cyclic pattern to interact with the 
raster line and its digitally written elements. As stated previously, the phenomenon occurs during an 
approach. The fields and the surroundings appear to be moving properly; however, the centerline of the run- 
way suddenly begins to slow down, stops, then reverses its direction, again stops, reverses direction and 
proceeds toward the aircraft. As the closing distance decreases, the runway gradually begins to have the 
proper appearance of motion relative to the surrounding fields. This phenomenon was corrected by dividing 
the centerline of a 13,500-ft runway into three submodels, each having a different number of visible segments. 
By combining these submodels with the eight priority steps ordered by the aircraft-to-runway distance, a 
specific mathematical temporal sequence was avoided, and, in doing so, the stroboscopic appearance of the 
slowing, stopping, and reversing of the runway centerline stripe was eliminated. 

These spatial -temporal sequences have not been a large problem in the calligraphic night-only and the 
simple day scenes of visual simulation systems in current use. The explanation may lie with their rela- 
tively low content of edges and objects. Therefore, the apparent motion of the pilot's viewpoint through 
space appears smooth and continuous. We may not have such success in the future if the scenes become more 
detailed or if we add texture to the scene in mathematically repeatable patterns. If the scene content 
becomes cyclic, nonuniform perceptions of speed will occur as stroboscopic effects, phi phenomenon, or 
flickering. 

There is no current systematic evaluation being made of the temporal-spatial perceptions induced by 
any of the existing visual systems. Even the simple scenes in use today can produce anomalies in perceived 
motion, if the aircraft speed is high enough and if the eye-to-surface distance produces the correct angular 
movement per unit of time. 
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5. THE IMPORTANCE OF THE VISUAL SCENE 

Since the first man attempted pictorial reproduction of objects, there have been judgments made of the 
quality of the reproduction. The most corimon one is usually in words akin to "that's a good (or poor) 
likeness." 

When we ask the question, how good is this visual simulation system or this display, we must indeed 
ask — Good for what purpose? The concept of image quality does not emerge until we have defined some task 
for the system to perform. The more limited and precisely defined we can make the task, the more accurately 
can the performance of the system be evaluated. Unfortunately, the ability of people to carry out visual 
tasks is related to more than just image quality - contrast and detection/recognition performance, observer's 
procedures, training, and time requirements are all factors. Nevertheless, if the essential elements of a 
given task or mission segment can be identified, it is reasonable to assume that a sufficiently realistic and 
accurate visual simulation system can be provided. The re.*;! challenge is in identifying the essential visual 
scene stimuli for a given flight control task and performance criteria and in determining the effect on these 
stimuli of closed-loop control situations with multiple inputs and control tasks. Having identified the 
essential visual scene stimuli, the next challenge is to establish precisely what is required for an 
"adequate" simulation. To what extent can one give up total duplication, of all the essential visual stimuli? 
What visual stimuli are needed by the pilot for making perceptual judgments regarding his position and veloc- 
ity? Are the same stimuli necessary in the simulator? AGARD's joint Working Group (Aerospace Medical Panel 
and Flight Mechanics Panel) on fidelity requirements of simulation for training purposes (Ref. 3) provided 
answers to some of these questions. 

Although a considerable amount of research has been conducted throughout the last century on visual 
space perception, the answers to these questions are not known. Graham gives a good account of the work 
performed prior to 1965 in his book (Ref. 18). Since then, a combination of neurophysiological and psycho- 
logical experiments have shown that many independent channels exist in the brain for the perception of space 
and motion. One recent discovery was made by Regan et al. (Ref. 62). It appears that certain neuron net- 
works in the visual cortex exist solely for the purpose of detecting the expansion of closed contours. This 
normally occurs when the object defined by the closed contour is traveling toward the observer. There is 
considerable survival value in being able to detect objects traveling toward one's head at high speed so it 
is quite likely that this aptitude has been genetically acquired. Regan attributes the ability of cricket 
and baseball players to hit balls thrown at them at speeds of almost 160 km/hr to these neuron networks. 
Apparently there are separate neuron networks that only fire when approaching objects are seen binocularly. 
This finding is in agreement with other workers who have found that stereoscopic information is processed 
by dedicated channels in the brain. Other workers in the field of visual perception have found similarly 
complex mechanisms for processing specific types of visual information. Butenandt and Grllsse, for instance, 
in 1968 (Ref. 63), reported that certain groups of cells in a frog's brain only respond when a convex area 
of shadow moves across the retina from right to left. 

There is little objective data on which to make judgments about how faithfully the real world must be 
represented for training or for researching various flight tasks and vehicle performance in a simulator. We 
know, for example, that skilled pilots have little difficulty when landing an aircraft at snow-covered air- 
ports devoid of most detail. Also, commercial transport pilots have little difficulty in landing an aircraft 
with a visual field-of-view limited to that of the wiper area of the windscreen. On the other hand, there 
are some military flying tasks, such as air-to-air combat and low-level, high-speed flight, in which the use 
of the resolution and image detail available in the real world is limited by the capabilities of the eye. 

Even though the eye is the fantastic instriment that it is, the flight environment contains much more informa- 
tion than can be fully utilized by the human visual system at any particular time. But an additional compli- 
cation to identifying the essential visual stimuli is the fact that the pilot does not necessarily see what 
he is looking at nor is he necessarily looking at what he sees. 

In any case, it is certainly clear that whether the visual simulation system is designed for training 
or for aircraft research and development, it will fall below the performance of the human eye and will pro- 
duce less than all of the real-world visual scene stimuli. Therefore, it is important that considerable 
effort be expended to determine the task or tasks for which the addition of visual cues will make the simu- 
lator most useful. It will also be Important to search for the psychoequivalency of visual cues which may 
make possible a dependable and useful hardware design. The visual simulation system should not try to 
duplicate the real world; it should simulate it to the extent necessary for the purpose for which it is to 
be used. 

Of the various options that are available to produce visual displays, no single system can suffice to 
represent the view from every type of aircraft, performing throughout its full mission capability, in every 
type of terrain and weather condition. Listed below are some of the tasks that require out-of-the-window 
visual information: 


Taxi 

Air-to-air combat 

VTOL operation 

Takeoff 

Ground attack 

Hover 

Climb 

Weapon delivery 

Nap-of-Earth flying 

Cruise 

Aerial delivery 

Oil rig support 

Descent 

Navigation 


Approach 

Terrain following 


Landing 

Maritime search 
Reconnaissance 
Air-to-air refueling 
Aerobatics 
Formation flying 



In addition to their classification by tasks as above, the requirements for the visual simulation system will 
be influenced by whether it is to be used for airline, general aviation, or military training, for special- 
purpose military studies of missions and tactics, or for research and development. 
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Some military aircraft and missions require the pilot to view the world from many different viewpoints 
and attitudes and to move through the environment at very high rates. Other missions, although less demand- 
ing in this regard, require that very large gaming areas be traversed. Some visual systems, such as the 
camera/model system, are very limited in terms of being able to represent extreme attitudes and large gaming 
areas. However, they can provide much scene content. Computer image generation systems, on the other hand, 
have little problem handling extreme attitudes and, with qualification, large gaming areas. However, they 
are very limited in scene content and complexity. For each of these tasks, different requirements will 
apply to the visual simulation system. Our limited current knowledge of the visual cues used by the pilot in 
performing these tasks, and our incomplete understanding of the mechanisms of perception, make the specifica- 
tion of the display requirement for each task a difficult one. Clearly, the field-of-view requirement for a 
straight-in approach is not as great as that needed for air-to-air combat, and the scene content of the dis- 
play is less in a night scene, or in bad visibility, than it is in clear, daylight conditions. The trade-off 
of one feature for another (such as field of view for resolution) is also difficult. Nevertheless, visual 
systems to perform all the above tasks have been constructed, and considerable benefits will be reaped if 
improved systems can be designed. 

It is important to determine which of these applications of the simulator are likely to result in the 
largest decrease in aircraft training time, the greatest increase in pilot proficiency and safety, or the 
greatest decrease in development risks. It is the job of the simulator engineer in designing simulation 
equipment and of the user in selecting simulation equipment to ensure that the most appropriate compromises 
in the available technology are made to satisfy the particular simulation requirement in the most economical 
manner. It is better to have installed a visual simulation system that does a few things well than to have 
installed a system that does many things poorly, one that may even result in negative training or in handling- 
qualities evaluations that could not be accepted at face value. Close cooperation is required between the 
person setting the simulator or training requirements and the person designing the visual simulation system. 

An experiment measuring the absolute value to training of different degrees of visual simulator fidelity in 
different training tasks is difficult to conduct, particularly if the experiment requires measuring transfer 
of training to the aircraft. However, the use of engineering development simulators may provide insight and 
qualitative judgments, even if an experiment that would yield quantitative results cannot be performed. It 
is worthwhile to investigate possibilities of psychophysical equivalency. If hardware limitations prevent 
the resolution, contrast, brightness, and accurate production of some essential visual stimuli, is it possible 
to use the same hardware to present to the simulator pilot a cue of equal discernibility? The use of part- 
task trainers by air carriers may provide an example to be followed. Teaching certain skills in a part- task 
trainer may be a more economical use of the flight simulator and may reduce the resolution requirements of 
the flight simulator visual system. 

A good example of a successful part- task trainer is the boom-operator's trainer developed by U.S. Air 
Force for the KC-135 aerial tanker. The visual system for this trainer provides a film background view at 
infinity, a TV image of the refueling aircraft positioned at the correct distance in space, and an image of 
the boom also correctly positioned In space. The correct monocular movement parallax cues and stereoscopic 

cues are provided to the operator, resulting in a high degree of training transfer. Anomalies such as being 

able to see through the refueling aircraft do not seem to detract from its training transfer. Other examples 
of part- task trainers are the calligraphic night visuals used for training takeoffs and landings at airports 
and the air-to-air combat simulators, such as the U.S. Air Force's Simulator for Air-to-Air Combat (SAAC).^ 
Calligraphic visuals offer a night landing image practically indistinguishable from the real world and have 
become almost universally accepted by the commercial aviation Industry. The simulator for training individual 
skills that will allow all aspects of flying, navigation, and weapon usage to be practiced, whether for heli- 
copters or fixed-wing aircraft, is a goal that may be neither achievable nor desirable. However, for aircrew 
and combat readiness training, a simulator encompassing the full mission may be essential. 

Pilots themselves exhibit many differences in their use of visual cues. The young student pilot may be 
able to learn faster if he is exposed to simple visual stimuli augmented with artificial cues. However, the 
real world contains a complex set of visual stimuli, and the pilot must learn to distinguish between the 
helpful and the misleading visual cues. Figures 21 and 22 in Section 3.3 show how easily the brain can mis- 
interpret visual information. They also demonstrate the feettoack process, which seems to reinforce the mis- 
perception of an object once it has been perceived incorrectly. It seems logical to assume that pilots should 
be exposed to complex visual scenes during advanced and continuation training. Whether a correct procedure 
exists for teaching pilots to extract relevant information from a confusing set of visual stimuli is unknown 
but those pilots with sufficient aptitude seem to learn somehow. On the other hand, because of the limita- 
tions of computer size and display technology, we are at present looking at repetitive patterns and models 

for adding detail to scenes. In doing this, we may actually destroy the effectiveness of some of our current 
simple scenes. 

Pilots with the same level of skill also seen to have different visual requirements. For example, with 
respect to the use of peripheral cues, Kraft (Ref. 64) tells the story of two senior pilots who tried one of 
the early calligraphic night visual simulation systems. That particular system had no side window displays. 

One pilot was able to make perfectly normal landings with no trouble; the other had a great deal of diffi- 
culty. The explanation offered by Kraft was that the latter pilot relied to a greater extent on peripheral 
cues than the first pilot, for neither pilot had any difficulty on the simulator that had side window displays. 
Pilots also react differently to color. An eye deficiency known as chromostereopsi s causes some pilots to 
estimate colored objects to be closer or farther away than they actually are. Using the correct colors in 
the simulator obviously has a certain amount of importance for these individuals. 


^The Simulator for Air-to-Air Combat (SAAC) was developed originally to support engineering and training 
research in air-to-air combat simulation. It is a two-simulator system for the U.S. Air Force's F-4 aircraft. 
The visual displays, mosaicked in-line, infinity systems, composed of eight pentagon-shaped channels of a 
dodecahedron, are synmetrically oriented about the front cockpit of each F-4 aircraft simulator to fill the 
field of view in the best way. The target aircraft imagery is generated via a camera and a rigid model, and 
the ground plane is generated via a contact-analog computer image generation system. The SAAC is operated by 
the U.S. Air Force Tactical Air Command at Luke Air Force Base, Arizona, primarily for air-combat training. 
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Pilots have a reputation for being adaptable to the characteristics of the simulator just as well as 
to those of various aircraft. They usually prefer to fly under visual meteorological conditions (VMC) 
rather than instrument meteorological conditions (IMC), and this preference also applies to their use of 
the simulator. The net result is that pilots undergoing training usually adapt very rapidly to whatever 
visual simulation system is being used and seem to prefer to continue using the visual system in training 
sessions, even when the picture quality is significantly degraded, rather than to train with no visual 
system. Whether the visual simulation system is providing positive training under these conditions is 
questionable. It is possible that the picture degradation acts as an extra visual workload and merely 
requires greater concentration on the part of the pilot to extract the relevant cues. The essential visual 
stimuli of the real world might be represented, but the perceptual workload might be entirely different. 

A similar situation exists when visual systems provide stimuli that are not present in the real world. 
Examples of this would be the degradation in focus that occurs on model-board simulation systems at low 
altitude (as a result of depth of focus of the optical probe) and the doubling of images on CGI systems at 
certain angular image rates. Pilots may learn to use these cues subconsciously. It is possible that extra 
visual stimuli such as these may have a beneficial effect on the overall performance of student pilots in 
the early stages of training but, if they begin to rely on these stimuli rather than on the intrinsic cues, 
the transfer of training to the aircraft may be reduced. Controlled experiments on transfer of skills from 
simulator training, similar to those performed by Lintern and Roscoe (Ref. 65), would be useful in assess- 
ing the importance of these synthetic cues. 

Another class of visual stimuli, usually described as distracting effects, comprises those stimuli that 
are due to aliasing or to the spatial quantization of the image by a CGI visual system. (Aliasing also 
occurs in model -board systems but is usually less distracting.) If these effects are not excessive, pilots 
become accustomed to them and the effects have little or no bearing on most training tasks. Habituation to 
this type of image anomaly may have significant effects in other areas. The attack helicopter pilot and 
copilot flying nap-of-the-Earth (NOE) in high- threat environments must continually be on the alert for 
unusual-looking objects and unusual movement in the scene. Severe aliasing and quantization manifestations 
can make every other edge appear to be a potential target or threat. Continued exposure to this type of 
scene may cause the pilot to become habituated to the "unusual" and degrade his target acquisition skills 
in the real world. The ability of a coninercial airline pilot to detect unusual objects in his flightpath may 
also be degraded by the same process. These ideas are purely speculative but warrant further investigation. 

The simulation of the landing approach is the most challenging of simulation requirements for fixed- 
wing aircraft and is similar in many respects to the simulation of the helicopter in NOE operations. The 
visual aspects for these flying tasks that are performed close to the ground are the most difficult to 
reproduce. Whereas the high-flying pilot can obtain adequate height cues from size and shape of known 
objects, the low-flying pilot must make use of texture and parallax. 

Pilot ratings and performance in the low-speed and hover flight regimes are extremely sensitive to the 
available visual and motion cues. For example, experienced VTOL and helicopter pilots were unable to hover 
with any precision with even the best attitude augmentation systems, when they used a single-window, camera- 
model visual system on NASA's Flight Simulator for Advanced Aircraft (FSAA). The reasons for this are not 
entirely clear. However, there is some evidence that this may not have been due entirely to the lack of 
peripheral cues. A research pilot hovered a UH-IH helicopter with increasingly reduced field of view with 
little or no reduction in performance or increase in workload. The same pilot indicated that the visual 
cues in the simulator were inadequate to hover the FSAA simulator with UH-IH dynamics. Explanations other 
than the lack of peripheral cues could have been an inadequate resolution or dead bands in the camera drive 
or possibly even deficiencies in the motion system. 

The pilot's ratings of control and display combinations are strongly influenced by the available out- 
side visual cue levels. Often, the simulator pilot is unaware that the source of a control problem is in 
the signals driving the visual display. 

In spite of their widespread use over many years, instrument-flying trainers have never been received 
enthusiastically by the whole piloting fraternity. The mbst frequent criticism is that the piloting tech- 
nique used by many pilots to achieve good results in the simulator can be different from that used in flight. 
Successful pilot training (and research investigations that use pilot evaluations) is only possible if the 
pilots accept the simulator, for what it can and cannot do. A visual display system adds greatly to this 
acceptance. 

The illusion of flight is only successful if the pilot can relate to the flight situation and divorce 
himself from the idea of sitting in a box, performing a stylized, though difficult, task. Acceptance of 
the simulator as an aircraft is greatly assisted by a visual display, whereby the pilot is made aware of 
the position, orientation, rotation, and translation of his aircraft in space. On takeoff, the pilot is 
made aware of the increasing speed and height of the aircraft from the visual display — an impression that 
is reinforced by the readings of the flight instruments. 

Although the objective of the Working Group was to "identify and define the physical parameters that 
characterize a simulator visual display," it is felt that certain characteristics inherent in the data base 
of the image generator play an important part in the acceptabi 1 i ty of a visual system. These items may not 
be measurable quantitatively, but their mere presence adds to the overall visual simulation system 
performance. 

Correct lighting effects are difficult to produce in a visual simulator. To produce the shadows charac- 
teristic of a bright sunny day requires collimated lighting of the model in a camera-model image generator 
and additional computation capacity and time in a computer image generator. Therefore, the representation 
of bright sunlight should not be attempted unless its effects are important to the task being simulated. 
Scenes without shadows are realistic representations of an overcast day and can be used successfully for' 
many tasks, particularly for the landing on a conventional airstrip, where the normal absence of vertical 
objects represents a shadowless envirorenent, even on a bright day. In air-to-air combat the directional 
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effects of the Sun can, however, be very important. The presence of a target aircraft is often first 
detected by its reflection of the Sun, even before the target aircraft itself can be resolved. This 
reflection can also be the first cue that the attitude or flightpath of the target is changing. Shadows 
can also be important in computer-generated representations of nap-of-the-Earth flight. The floating 
appearance of trees or other objects of unknown size and range is largely eliminated when they are tied 
to the ground by shadows. 

The incorporation of landing light effects into the night scene of a simulator visual system is also 
important. Although the number of lights that can be displayed simultaneously is a means of judging image- 
generator capacity. It is also important to investigate the possibilities that the image generator offers 
with regards to the form in which these lights are displayed. Two aspects need to be addressed here, one 
being the capability of the image generator to compute light directionality and rotating or flashing lights 
and the second the attenuation of light intensity with range and its modulation by fog. Both characteristics 
are considered highly desirable in a simulator environment. 

The effects of a solid cloud layer (i.e., blockage of direct sunlight), zero visibility in clouds, and 
unlimited visibility above clouds are desirable in a daylight visual system. Clouds may also be useful in 
relation to requirements for picture detail, for the time spent flying in clouds can be used to read new 
data bases from a mass storage device, thus eliminating the need for processing large quantities of data 
that can never be seen (i.e., calculation of ground detail when flying above clouds). Distant clouds can 
also give the pilot important cues of heading and roll change. 

Fog and generally reduced visibility effects are key features in some types of flight simulator train- 
ing, One must differentiate between two effects. The first is the change in color of an object seen through 
a layer of fog, which will slowly tend toward the color of fog with distance, depending on the density of the 
fog. This is an entirely subjective value and needs to be adjusted to suit the displayed scene (night, dusk, 
or daylight). The second effect is the decrease in brightness of a displayed object as a function of range. 

It roughly follows a "one-over-di stance-squared" law and, in order to avoid Mach banding, needs to be simu- 
lated accurately and continuously. Both effects tend to reduce the saturation of the color and the contrast 
of a displayed object with range, so that the object disappears slowly and continuously in uniform fog. It 
must be noted that for correct simulation of fog the three-dimensional character of a fog layer must be simu- 
lated accurately. This can only be achieved by testing each displayed picture element (the smallest computed 
picture unit) for its range to the observer's eye. If this procedure (which takes up a lot of computing time) 
is strictly followed, then curtain effects - as can be observed in some electromechanical closed-circuit 
television/model -board visual systems -will be avoided. Fog in the real world is not always uniform, and 
when it is not, it follows several different patterns. Therefore, a simulation that is subjectively acceptable 
to one simulator pilot may be considered unacceptable by another. 


A digitally computed visual scene will display discrete changes in color, brightness, and position of 
objects from one scan-line segment to another. This enhances the artificial appearance of some pictures. 
There are several means - some of them covered under "picture anomalies" of reducing this effect. One is 
called "smooth shading" and is based on the calculation of color and brightness changes on a picture element 
level. To illustrate this procedure, a color change within one scan-line, with and without smooth shading, 
is shown in Figure 58. 


Rather than introduce an abrupt color change at 
picture elenent Xg, the color is incrementally 
changed for each computed picture element starting 
at Xa and ending at Xq. This allows a smooth 
transition of colors without any detectable discon- 
tinuity. This technique is important in the simu- 
lation of fog, horizon haze, structured cloud tops, 
tire marks on the runway, and round objects, ,It 
may be debatable whether this time-consuming proce- 
dure needs to be incorporated into a computer image 
generator, but practical experience has shown that 
it enhances picture fidelity and, therefore, overall 
visual system acceptability. 

The proper occulting of surfaces and lights as 
the aircraft moves to any position within the modeled 
area also should be part of the overall image gener- 
ator capabilities. 



Fig. 58 Smooth shading technique 


In summary, the problem of the visual simulation system designer is one of striking the optimum balance 
between available technology trade-offs and performance requirements (i.e., the degree to which the technology 
should be pushed), the degree to which the system perfonnance can be compromised without significantly impair- 
ing the utility of the system, the technical skills of the available operating staff, and the cost of the 
resulting system. 
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6. FUTURE HARDWARE TRENDS 

The primary limitation in satisfying the more demanding visual simulation requirements is in the visual 
display. Computer image generators can provide full field-of-view imagery at relatively high resolution in 
color. The limitation on the image generator is more economic than technological. However, display tech- 
nology is severely limited in terms of being able to provide high resolution simultaneously with wide fields 
of view. Color at relatively high brightness levels for wide fields of view is also beyond the state of the 
art. Another display limitation is the inability to provide a relatively wide field-of-view imagery to two 
or three crewmembers who are several feet apart without parallax or positional errors. 

Developments are currently under way to provide improved visual display capability for the single-viewer 
fighter/attack and the multiviewer transport/bomber aircraft simulators. Several television projector develop- 
ment efforts are directed toward providing high-resolution, high-brightness, color-display input devices. 

These projectors have application to both the single and multi viewer displays for both real -image and infinity- 
display types. There are four basic types of projectors currently under development: CRT, liquid-crystal 
light-valve, Tifus light-valve, and laser. Also, improvements are continually being made to the oil -film type 
of light- valve projector. Some development efforts in CRTs and CRT displays are currently being pursued. 
Miniature high-resolution monochrome and color beam-penetration CRTs are being developed for helmet-mounted 
displays and a full -col or calligraphic display has recently been developed that uses a high-resolution, 
shadow-mask CRT. 

6.1 Projectors 

One CRT projector under development utilizes a unique arrangement of three CRTs mounted on three 
vertical sides of a cube containing dichroic beam splitters and a special optical matching and cooling liquid. 

A sketch of the projector is shown in Figure 59. The 
projector combines the imagery from the three primary 
color tubes and projects it through a single acrylic 
lens. A 1023-TV-line version of this projector is 
being developed by the Electronics Systems Products 
Company for the U.S. Air Force. Through cooling of 
the faceplates by the liquid-filled cube, light output 
of the order of 500 jim should be possible. Higher 
resolution should be possible with some reduction in 
light output. This projector is attractive from the 
standpoint of its simplicity, small size, and low cost. 


The liquid-crystal light-valve (LCLV) projector 
was developed by the Hughes Aircraft Company during 
the late 1970' s. It uses a projection lamp to supply 
luminous flux, and accomplishes projector light modula- 
tion by means of a liquid-crystal device, which itself 
is modulated by a source of "writing light." For simu- 
lation application, the writing light is supplied by a 
small CRT, which is fiber-optic coupled to the liquid 
crystal and modulated by a video signal. The principle 
of operation of this projector is shown in Figure 60; 
the fundamental function of the LCLV is illustrated in 
Figure 61. Full -color operation can be achieved by field-sequential operation or by using red, green, and 
blue "channels," each consisting of a CRT and LCLV which are optically superimposed by means of dichroic 
mirrors. This projector has the potential for great flexibility in operational characteristics, such as 
brightness and resolution, since a proper combination of illumination lamp, liquid-crystal material, photo- 
conductor, and CRT can theoretically be achieved for any desired application. A 1023-TV-line color LCLV 
projector was recently developed for the U.S. Air Force which provides a 820.cd/m^ polarized light output. 

A follow-on version is being developed which is to have a 1360-TV-line background resolution with an equiv- 
alent 2700-TV-line inset raster resolution capability. 
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Fig, 60 Principle of operation, LCLV projector 
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Fig. 61 LCLV fundamental functions 
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The Titus tube light-valve projector was developed by SODERN (France) in the mi d- 1970 's. Its principle 
of operation, as shown in Figure 62, is similar to that of the LCLV projector in that it utilizes an external 
projection lamp, a polarizing/analyzer prism, and a device that accomplishes projector light modulation. 
Instead of an LCLV, a Titus tube is used to modulate projector light. The Titus tube is an electron-beam- 
addressed Pockels-effect imaging device using a KD 2 P 04 plate operated just above its Curie temperature in the 
reflection mode. The Titus light-valve operates on the basis of the variation in birefringence in a crystal 
when an electrical field is applied through the crystal in the same direction as the light propagation. The 
Titus-tube light-valve projector is capable of high luminous output, exhibits no raster line structure or 
flicker, possesses random-access image-writing capability, and has the capability to compensate for distor- 
tion. A Titus-tube color projector is currently being built by SODERN for the U.S. Air Force for simulation 
application. It is to provide a 1080-TV-line resolution capability with a polarized light output of 2700 m. 

Oil -film light-valve projectors, developed in the late 1950* s and early 1960's, have been used in 
numerous simulator visual systems. Improvements in resolution and light output have been made in this type 
of projector over the years. A color, 1023-TV-line version available from General Electric will provide a 
light output of 650 tm. 

The laser projector utilizes neither a kinescope phosphor nor a projection lamp to supply luminous flux. 
Instead, a laser supplies luminous flux in the form of a very bright beam. The laser beam is deflected in 
the vertical and horizontal directions by means of optomechanical or electrooptical scanning devices to write 
out a raster on a screen. At the same time, the laser beam is amplitude-modulated by video information in 
synchronism with the deflected beam's position within the raster, thus producing a bright projected image. 

If full color is desired, multiple lasers of different color can each be individually amplitude-modulated to 
produce red, green, and blue information, and the multiple laser beams can then be deflected by a common 
scanning system. Prototype projectors utilizing this principle of operation have been built with limited 
success since the late 1960‘s. A recent development program conducted by the American Airlines/Redifon 
Simulation, Ltd., under joint sponsorship of the U.S. Army, Air Force, and Navy has resulted in the develop- 
ment and successful demonstration of a very promising prototype two-color scanned-laser projector. A laser 
camera scanning a model -board is used to generate the imagery. This system provides a continuous 180° hori- 
zontal by a 60° vertical field of view. A scanned laser display is shown representatively in Figure 63. A 
similar scanned-laser visual system has been developed by Singer Librascope. It provides a narrower field 
of view, but is a full three-color system. Although scanned-laser projectors must be further developed 
before they can be put into production, they offer great promise for fulfilling relatively wide field of 
view requirements with a single projector. Because of the size and location of the scanner, significant * 
increases in the vertical field of view do not appear feasible. 
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Principle of operation, Titus tube-light 
valve projector 


Fig. 63 Visual simulation display utilizing scanned 
laser projector 


6.2 CRTs 


Visual simulation has stimulated the development of CRTs of extreme sizes. The world's largest CRT, 

36 in. in diameter, was developed during the early 1970* s for mosaicked in-line infinity displays used on 
the Advanced Simulator for Undergraduate Pilot Training. At the other extreme, small high-resolution CRTs, 
approximately 1 in. in diameter, have been developed for helmet-mounted displays. 

A metal -funnel version of the large 36-in. CRT is currently under development; its development was 
undertaken primarily to reduce costs. The cost of the metal-funnel version is estimated to be less than 
one-half that of an all -glass version. There is also a side benefit of safety - the metal -funnel CRTs 
are much safer to handle during manufacture as well as during installation and replacement. These CRTs 
are being developed by Thomas Electronics under U.S. Air Force sponsorship. The intended application is in 
tactical air-combat visual simulation systems. 


A high-resolution, high brightness, 1-in. CRT for use in helmet-mounted displays is also being devel- 
oped. This CRT is to be capable of 1200-2000 TV-line operation while producing 1000-1400 cd/m^ highlight 
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brightness. This resolution and brightness capability is the result of improvements in high-efficiency, 
small -grain phosphor and improved electron-gun and deflection-coil design. It is to be used with wide-angle, 
binocular helmet-mounted display optics providing a field of view of approximately 60° vertical by 135° hori- 
zontal. The CRT is being developed by Thomas Electronics with the participation of several other vendors 
under U.S. Air Force sponsorship. 

A small, beam-penetration CRT is also under development for providing a color capability for helmet 
displays. The CRT is two-color, red and green, and is of the layered phosphor type. 

High-resolution shadow-mask CRTs are being used to achieve higher resolution, full-color presentations, 
both raster scan and calligraphic. A recent development by Evans and Sutherland has provided a shadow-mask- 
type CRT display that can be driven calligraphically. This capability is the result of new developments in 
drive electronics, in particular deflection circuitry, and the use of existing high-resolution shadow-mask 
CRTs. 


6.3 Area of Interest 

Since the straightforward or brute-force approach to improving the resolution throughout the total field 
of view is apt to require significant advances in the state of the art and to result in very expensive sys- 
tems, other more innovative approaches have been considered, some of which are under development. Some of 
these techniques, such as raster inset or target superposition, typically called area-of-interest techniques, 
are intended to provide image resolution and detail only where required in the scene, with lower resolution 
throughout the background area. There are three basic approaches to achieving this, and there are several 
variations of each. The first, called target slaving, has the high-resolution area depicting the target 
controlled by the location of the target itself. The second, called head slaving, takes advantage of the 
instantaneous field of view (i.e., the field of view with the head stationary) being less than aircraft 
field of view. This approach requires head-position sensing. Fairly reliable head-position sensors are 
currently available. The third, called eye slaving, takes advantage of the falloff in resolution capability 
of the eye as a function of angle off the visual axis of the eye. In this approach, the visual system reso- 
lution would attempt to match the performance of the eye - a demanding task only in matching the eye's per- 
formance over the relatively small foveal area. This approach requires information concerning the line of 
sight of the eye, in order to appropriately position the high-resolution part of the scene where the pilot 
is looking. This presents other challenges with respect to eye trackers and head-position sensors. Since 
eye trackers will require significant improvement before they can be made operational, the approach of 
target-controlled, high-resolution areas with head-position sensing to eliminate the requirement to generate 
imagery that falls outside the instantaneous field of view may be the most promising approach for the near 
future. 

6.4 Display Optics 

Improvement in optical design techniques and the development of techniques for producing both large 
refractive and reflective plastic optics are resulting in Improved display performance and reduced display 
weight and cost. Plastic mirrors have recently been developed by the McDonnell Douglas Electronics Company 
for classical mirror-beam- splitter infinity optics systems which reduce weight and offer potential for larger 
continuous field of view. Techniques are being developed by Redifon of England, under U.S. Air Force sponsor- 
ship, for producing 5-m-radius, plastic spherical mirrors. These mirrors are to be used in a multi viewer 
display for the transport/bomber-type of aircraft simulator. If successful, this development will provide 
the technology to produce an infinity display with a continuous field of view of 60° vertical by 180° hori- 
zontal with a viewing volume 1.5-m wide, 1-m fore and aft, and 0.5-m deep. An approach to providing a 
multi viewer display, using improved large plastic refractive lenses, is being pursued by the Electronics 
Systems Products under U.S. Air Force sponsorship. This approach offers potential for a more compact and 
less expensive multi viewer display. 

More exotic approaches to using Fresnel refractive and holographic reflective optical elements have 
been pursued recently. The large Fresnel experimental lens is still under development, having been delayed 
by the unavailability of diamond turning machines able to handle l-i.25-m plastic blanks. The principal 
advantage of the Fresnel lens is its projected lower cost and weight. The holographic reflective optics 
also offer very significant cost and weight advantages. Good quality 81-cm monochrome holographic lens 
analogs have been developed recently. However, going to the larger 122-cm and 152-cm holographic optics 
required for mosaicked in-line infinity displays requires a significant investment in new facilites to 
handle the larger holograms, and the necessary funding has not been available. 

Fiber-optics cables are being considered for use as coupling elements in several types of visual sys- 
tems. They are used in the scanned-laser visual systems just described to couple laser light to the projec- 
tion system scanner. The U.S. Naval Training Equipment Center is developing a visual system that is essen- 
tially a helmet-mounted projector. It is to use a laser as a light source; the light is modulated and 
deflected horizontally off the helmet. The light is' then coupled to the helmet, through a fiber-optics rib- 
bon, where it is deflected by a small vertical scanner mounted on the helmet. This produces a scanned-laser 
image on a high-gain dome screen whose location is a function of head position. One significant advantage 
of this approach, which results from using fairly low light levels and high-gain screen material, is that 
the projected light striking the cockpit and instruments is so dim that it does not need to be masked. 

A color multiplexing technique, using fiber optics developed by American Optical Corporation, has been 
considered for helmet displays that use image-input devices off the helmet. This technique uses a wedge- 
shaped prism at the input end of the fiber bundle to spread each picture element over many fibers. An 
achromatic image is formed at the output end, using an identical prism. This technique not only removes the 
fiber structure from the image, but increases the limiting resolution by as much as a factor of 2 over the 
same size cable without color multiplexing. Another interesting property of this technique is that the 
resolution of the cable is no longer a function of the number of fibers in the cable, but depends instead 
on the ratio of the individual fiber diameter to the overall cable size. 



64 


6 . 5 Hel met-Mounted Di splays 

The ideal visual system as envisioned by many over the past decade is one that is small and light 
enough to be carried on the pilot's helmet. Helmet sights, using small CRTs and optics, are in operational 
use; several efforts are under way to extend this technology to provide a limited capability helmet-mounted 
visual display. These devices typically utilize 1-in. CRTs as image input devices which are viewed through 
optics that collimate the imagery and superimpose it over the cockpit scene. A 1-in. two-color beam- 
penetration CRT has recently been developed for McDonnell Douglas Electronics by Thomas Electronics to 
provide a limited color capability for such a display. These displays are limited in both field of view 
and resolution. 

Designs have been developed by Farrand Optical under U.S. Air Force sponsorship for optics to provide a 
larger field-of-view binocular helmet display. These optics provide a field of view of about 60® vertical 
by 135® horizontal. Such optics, used with small CRTs, will provide a medium display resolution satisfactory 
for only a limited number of flight tasks, such as takeoff and landing, formation flight, and aerial refwling. 
Other concepts have been proposed that would couple imagery to the display optics via fiber optics rather 
than being limited to input CRTs mounted on the helmet, thus offering potential for improved resolution and 
brightness as well as full color. Multiple-level resolution display inputs with eye tracking have also been 
proposed in order to provide a display resolution that conforms approximately to the static resolution of 
the eye. An eye tracker would enable the highest resolution part of the display to be continually positioned 
according to the line of sight. 

6.6 Outlook for Displays 

Through the development of various projector devices, higher resolution and higher brightness color 
displays will be available in the future. These devices, used in conjunction with mosaicked infinity optics 
and real -image domes will be able to provide wide fields of view with improved resolution and brightness as 
well as color. The cost of such displays will be rather high, which will continue to stimulate the develop- 
ment of area-of-interest approaches. The target-slaved approach, with head sensing to reduce the instanta- 
neous field of view, seems to hold the most promise for the near future. 

The development of wider field-of-view optics as well as the proposed use of fiber optics input has 
greatly improved the potential of helmet-mounted displays. Such displays can be expected to be utilized 
in a variety of simulator applications including ground-based, air-combat, part-task trainers as well as 
both ground-based and airborne engineering research simulators. 

At a still later date, it seems plausible that displays that are able to utilize digital information 
directly from a computer image generator will be developed. Such displays would be attractive from the 
standpoint of eliminating the video link and the problems associated with broadbanded high-power video 
equipment. Such displays could take the form of large arrays of display elements or some type of matrix- 
addressed light-valve projector. 

It might be noted that the direction of display developments will be influenced by their potential 
use in the entertainment industries. 

6.7 Computer Image Generation 

The capabilities of both calligraphic and raster- scan CGI systems have improved dramatically during 
the past 5 years. Calligraphic systems, since their introduction during the early 1970* s, have evolved 
from point-light, night-only systems to night/ dusk and recently to night/dusk/daylight systems. The 
recently developed night/dusk/daylight systems are essentially hybrid calligraphic, with surfaces being 
generated and displayed with a full field-of-view calligraphic raster and lights being handled in the con- 
ventional manner. The principal breakthrough that permitted this step into the daylight CGI world was the 
development of a high-resolution, "full" color, shadow-mask calligraphic display. A calligraphic color 
projector has also been developed recently that will allow further exploitation of the calligraphic tech- 
nology for various simulator applications. Future trends seem to be toward increased surface generation 
capability and improved displays with brighter imagery in full color and more flexibility for various 
simulator applications. 

Current generation raster- scan CGI systems offer many advantages over other previous image-generation 
techniques, such as being able to handle large fields of view, providing relatively high resolution, allow- 
ing flexibility in changing the environment or data base, allowing flexibility in viewpoints and view windows, 
and providing infinite depth of field. The primary limitations have been in the ability to generate image 
detail and to eliminate artifacts associated with the generation process. However, the many advantages 
have apparently outweighed these limitations and have led to the rapid acceptance of CGI and its recent 
exploitation for virtually all types of visual simulation applications. The limitations are very salient 
and have simulated considerable development directed toward increasing image detail and content and reducing 
spurious information and artifacts in the imagery. Some vendors have concentrated on increasing image- 
generator capacity and others have concentrated on improving image quality. 

The straightforward approach to providing greater image detail through building higher-edge- capacity 
systems has been pursued. by most vendors. Systems with capacities of the order of 8,000 edges or several 
thousand surfaces are currently available, and some vendors are discussing or developing systems that 
reportedly will have capacities of the order of 100,000 edges. Figure 64 (from Ref. 66) indicates the 
increase in system edge capacity over the past 15 years and the projected rapid increase in the future. 

Advances in very large-scale integration promise rapid advances in hardware performance and reductions in 
cost. 


Another technique that has been incorporated in most current systems and used effectively to increase 
detail and improve the efficiency of CGI systems is level-of-detail (LOD) control. LOD control essentially 
attempts to keep the 'number of processed edges at a high level near the upper capacity of the system. More 
detail in the form of edges is added to the part of the scene being approached by the viewpoint and is removed 
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from the part of the scene that is receding from the 
viewpoint or falling outside the field of view. Although 
these techniques have been used effectively in current 
systems, there are times when the LOD changes at dis- 
tances from the viewpoint where it is quite noticeable 
and causes a distracting "pop in" and "pop out" of 
detail. Improved level-of-detail management techniques 
can be expected in the future to minimize some of these 
distracting effects. In addition, techniques are being 
developed by some vendors to blend intensity gradually 
between levels of detail to eliminate distracting 
instantaneous changes. 

Area-of-interest approaches are compatible with 
CGI techniques and can be expected to be used exten- 
sively in future tactical airTCombat simulator applica- 
tions. Techniques are being developed that will pro- 
vide changing level of detail and resolution as a 
function of target, head, or eye position. 

Another technique that is being pursued by most 
vendors to increase image detail without resorting to 
ever-increasing edge-generation capacities is texture 
generation. Several texturing approaches have been 
or are being developed; they range from rather stylized 
patterns to texture that is very characteristic of 
natural and man-made cultural objects. More advanced 
three-dimensional texturing techniques, which will 
provide parallax among features of and objects within 
the texture, are currently under development. 
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Progress is being made in improving image quality 
through the development of improved antialiasing tech- 
niques. Most current schemes essentially compute 
individual pixel intensities on the basis of averaging 
the intensities of all surfaces within the pixel. This 
usually involves the computation of subscan-lines and 
subpixels. Although these techniques work reasonably 
well, they do not completely eliminate all aliasing problems, especially in dynamic imagery. In addition, 
although these techniques are fairly straightforward conceptually, they are costly in computation. The 
Improved techniques involve spatial filtering at the pixel level and result in images that have high subjective 
resolution and sharpness, with the higher spatial frequencies appropriately attenuated. Further details on the 
spatial filtering techniques are available in References 57 and 58. It is expected that these techniques will 
be used in many future systems. 


Future military simulation requirements for both tactical and strategic missions involve the simulation 
of a high percentage of natural terrain over extremely large gaming areas. Edge and planar surface techniques 
are not optimally suited for representing most types of natural terrain. The large gaming areas require enor- 
mous amounts of digitized source data, which is being provided or will be provided by the U.S. Defense Mapping 
Agency (DMA). To produce the on-line CGI system data bases from these source data requires the use of highly 
automated transformation programs because of the enormous amount of data involved. Even though the process 
is highly automated, it is very time-consuning and does not usually result in a completely satisfactory 
on-line data base without some manual rework. The above future requirements and the associated simulation 
problems have stimulated interest in the development of CGI approaches that are better suited for simulating 

natural terrain and that are amenable to more direct use of DMA source data. At least two companies, 

Honeywell and Grunman, are pursuing such approaches. Some of these approaches involve the generation of 

curved surface fits to the terrain data, using bicubic patches or quadric surfaces. These approaches are 

also compatible with texture mapping. Although there may be some simple nonedge-based systems in the field 
within the near future, the more sophisticated nonedge-based systems are bnly in the preliminary development 
phases and may not be available until the late 1980' s. 
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7. RECOMMENDATIONS FOR RESEARCH 

Current and foreseeable technology will not produce images of the outside world that represent, in all 
aspects, the visual scene observed by pilots. Faithful reproduction of the real world is not only tech- 
nologically unfeasible but would be prohibitively expensive even if it were feasible. Moreover, experience 
with the use of existing simulators indicates that considerably less than duplication of the real world has 
been satisfactory for many simulator applications. However, in the absence of firm data on requirements, 
the tendency has been to strive for higher fidelity and, possibly, to err on the side of overrealism. Past 
approaches to establishing simulator requirements typically have tended toward duplicating the flight vehicle 
and its environment. These approaches have led to more and more sophisticated and costly simulators. Con- 
sequently, the most pressing need for research in all areas of simulation, and especially visual simulation, 
is research that will lead to a better definition of requirements. Little objective data exist on precisely 
what is required for training or researching various flight tasks and vehicle performance in a simulator. 

The problem is to identify the visual cues used for a particular task and then to translate from func- 
tional definitions, such as orientation or speed cues, to the characteristics of the visual simulation system, 
such as scene content, luminance, texture, contrast, and color. 

The question "What cues do pilots use?" seems to be asked in one form or another at every simulation con- 
ference, but it is unlikely that a strictly theoretical approach will provide the answer. The enormity of 
this question becomes more and more apparent as ^one reads about the extensive research being performed cur- 
rently in the area of vision and visual perception. The only promising approach for evaluating the usefulness 
and necessity of the various visual stimuli available to the pilot would seem to be through transfer-of- 
training experiments and performance-evaluation experiments in real aircraft. These may be initiated by 
simulator engineers and users but should be performed with the participation of experienced experimental 
psychologists to ensure that the results will be valid. 

One of the lower-priority objectives of the Working Group was to relate the physical visual system 
parameters to the visual perception of the observer. The problem was frequently discussed, but it became 
evident that any major effort in this direction would detract from the primary objective. It now seems 
appropriate, however, having analyzed the physical parameters of visual simulation systems, to attempt this 
task. Certain relatively simple control tasks that require visual inputs seem to lend themselves to mathe- 
matical analysis and experimental verification, A control model might be formulated that would make it 
possible to predict a pilot's performance in a simulator and to compare it with his performance in a real 
aircraft. The pilot may not need to use the same visual inputs or cues in the simulator as he does in the 
aircraft, but a basic assumption would be that he needs to be able to control the simulator with the same 
degree of precision and with the same control strategy as he controls the aircraft, using visual cues as 
the only source of information regarding the velocity and orientation of the aircraft. This assumption 
would seem to be reasonable for both training and research simulators. The analysis of simple control tasks 
would not only be useful in itself but would provide insight into how the more complex visual tasks could be 
analyzed. 

Some of the more successful simulators have been based on requirements determined from thorough analyses 
of the tasks to be trained or researched in the simulator and the information necessary to perform the tasks. 
Some of these requirements have been determined from observing what the pilot must do in the aircraft to per- 
form the task successfully, for example, the field of view used in executing an approach and landing or the 
typical acquisition distances for targets in an air-combat engagement. A variation of this approach involves 
reducing the fidelity of the information received by the pilot in an actual flight situation to the point of 
being able to determine where his performance is adversely affected. For example, special glasses that reduce 
the effective resolution of the real-world visual scene as viewed by a pilot could be used to determine what 
minimum resolution is required by a skilled pilot to perform a particular flight task successfully. This 
same approach could be used to determine requirements for color, field of view, brightness, and, possibly, 
image detail for future simulators. 

Another type of information that has not been fully exploited in the design of simulators and the 
determination of fidelity requirements is psychophysical data on human sensation and perception. Such data 
have been used in the past to make decisions on television refresh rates, interlace techniques and color 
approaches, and these decisions have been universally accepted. Other simulator visual system issues, such 
as brightness, contrast, resolution, detail or scene content, and a rea-of- interest approaches, can perhaps 
be resolved to a high degree of confidence through further use of psychophysical experimental data. 

Not very much is known about motion perception although many manifestations of the phenomenon are 
observed. Motion perception is so sensitive in humans that many variables affect it: whether the eyes are 

moving or stationary, whether the motion occurs on a textured or plain background, and the duration of 

observation are examples of factors that affect the measurements. In addition, many of the variables, such 
as content, are resistant to being dimensional ized and others, such as viewing distance, have an uncertain 
status. A variety of experiments, including those by Bouman and van den Brink (Ref. 67), Pollock (Ref. 68), 

Liebowitz (Ref. 69), Mashhour (Ref. 70) and Kolers and Pomerantz (Ref. 71), have not yet succeeded in explain- 

ing completely the perception of motion. The literature on temporal aspects of perception raises two funda- 
mental questions. The first is. Does the human visual system ever note velocity directly? The data suggest, 
to the contrary, that some of the constituents of velocity, such as time, distance, and intermittency, are 
more important to the judgment than is velocity itself. The second question is. Does the human visual system 
process spatially separated and contiguous flashes in different ways? The data suggest that the human visual 
system constructs the two perceptions of motion quite differently. The immediate need is to know the spatial - 
temporal range over which the perception of continuous motion in visual simulation systems is acceptable. 

These data also indicate a need to examine the perception of motion in the context of man moving through space 
as well as objects moving past a stationary man . 

In CGI systems, more study is required of the effect on visual perception of the various sporadic 
phenomena in the display scene, classified under the general term "aliasing." In camera/model -board systems, 
distortion of the video image and a reduction in contrast at the edges of objects occur as a result of the 
time taken in producing a complete video frame. The magnitude of these effects is a function of the velocity 
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of the image across the face of the display. Psychophysical research is required to determine whether 
these effects (which are only noticeable at high velocity) affect visual perception. 

Probably the principal reason that psychophysical data have not been brought to bear on the simulator 
design problems in the past is that the sensory and perceptual information has not been available in a form 
readily usable by simulator design engineers. In view of the potential of such data as a valuable source of 
simulator design information, a concerted effort should be made to transform relevant existing psychophysical 
experimental data into a form in which the data would be readily usable by the simulator design engineer and 
to develop other psychophysical data essential for answering important simulator design questions. 

The preceding discussion stated that research is needed to identify the visual stimuli used by a pilot, 
in particular flying tasks, and to determine the relationships between these visual cues and the character- 
istics of the visual simulation system. However, regardless of the answers to these fundamental issues, 
there are known deficiencies in the technologies of visual simulation systems that need to be addressed in 
research and development programs. It is known that the state of the art of visual simulation is not adequate 
to satisfy the requirements of certain demanding military flight tasks, even though those requirements are not 
yet precisely known. For example, research and development are still required in the visual display and image- 
generation areas, with efforts in the visual display area being the higher priority, since this is where the 
greater deficiencies lie. 

The development of display technology that can provide higher resolution and brightness simultaneously 
with large fields of view is urgently required. This development work would involve improvement in compo- 
nents such as display optics, screens, and projectors. Although color is still one of the many unknowns 
with regard to visual simulation requirements, there is justification for continuing the development of 
color-vTSual-di splay technology with higher brightness and resolution. Part of the justification is the need 
to present color-coded information such as is provided in runway and aerial -refueling director lights. There 
is also evidence that, under certain conditions, color aids in target identification. 

Improved visual system brightness and resolution approaching the capability of the human visual system 
need to be developed for use in fighter/attack aircraft simulators for certain training and tactical weapon 
system mission assessments. For such applications, capability of displaying targets and threats at real 
detection and recognition ranges appears essential. This capability would be required along with a full 
aircraft visual field of view. In order to provide this capability, television projector development efforts 
need to be emphasized. These efforts need to concentrate on providing a display resolution capability 
approaching 1 arcmin at brightness levels greater than 30 cd/m^. In addition, infinity optical mosaic and 
real -image dome display techniques need to be developed to provide full-field-of-view displays at reasonable 
costs. Development of projector technologies, such as the various light- valve types, including oil -film, 
liquid-crystal, and solid-crystal, needs to continue. 

The visual displays in the fighter/attack simulator are unique in that, typically, the out-of-the-window 
visual information needs only be provided to a single crewmember. On the other hand, visual displays for 
the bomber/ transport simulators ideally must provide the same visual scene to several crewmembers, displaced 
3-4 ft, without parallax errors, gaps, and other distortions. Such displays, which employ collimating optics 
of either the reflective or refractive type, are called multi viewer displays and require rather large optical 
elements and special screen material. Such developments will depend on new manufacturing techniques for pro- 
ducing large, high-quality optical elements at reasonable costs. 

Area-of-interest techniques are intended to provide high image resolution and detail only where required 
in the visual scene, with lower resolution throughout the rest of the visual area. These techniques offer 
great potential for reducing the cost and complexity of visual systems by reducing the required number of 
image- input devices or image-generation channels or both. However, such techniques require information on 
either head position or head position and line of sight to control the location of the area-of-interest 
region of the display. Fairly reliable head-position sensors have been developed; however, oculometers for 
precisely determining the line of sight need further development. 

It has generally been assumed for area-of-interest types of displays that the resolution in the visual 
periphery need only match the static acuity of the human visual system. However, targets in motion that are 
smaller than those that can be represented at the minimum static resolution of the display may be quite 
detectable in the real world. For this reason, an important area of research is to determine the required 
peripheral resolution for depicting small dynamic imagery in area-of-interest displays. An associated issue 
is to determine the acceptable and optimum ways of insetting and blending the region of high detail and 
resolution into the lower detail and resolution background. If the boundaries are too apparent, they are 
not only distracting but they provide information about the target area that is not available in the real 
world. Techniques for blending the areas of different resolution must be developed before the area-of- 
interest concept will be completely satisfactory. 

The development of helmet-mounted display technology has progressed significantly during the past 
5 years. Optical designs have been developed that allow a relatively large field of view to be provided to 
both eyes with image overlap. The primary limitation, as with most visual displays, is in the resolution of 
the imagery that can be presented. With the concept of mounting small CRT's on the helmet, there is a trade- 
off between resolution and weight on the helmet. Weight and Inertia constraints have limited the image-input 
devices that can be mounted on the helmet to two 1-in. CRT's. Typical resolution achievable with this approach 
is no better than 5 arcmin per TV line. Although this may be satisfactory for some flight tasks, it is not 
acceptable for many tactical air-combat flight operations. 

In order to make helmet-mounted displays feasible for air-combat visual simulation systems, concepts 
and techniques for inputting higher-resolution information must be developed. In one such concept, the use 
of fiber optics to couple high-resolution image-input devices to the helmet display optics, thereby avoiding 
the problems associated with mounting the image-input devices directly on the helmet, is proposed. This' per- 
mits an area-of-interest approach with multiple image-input devices providing resolution capability matching 
that of the htman visual systan. This concept has great promise and could revolutionize future visual 
displays. 
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Computer image generation systems are becoming more and more complex as the requirements for image 
detail, resolution, and field of view increase. However, most of the system architectural approaches in 
use today were designed for low-edge-capacity systems. With the demand for ever increasing edge capacity 
and the addition of new system primitives, such as circular features and texturing, it is appropriate to 
develop totally new architectural approaches more suitable to high-edge-capacity systems employing very 
large-scale integration (VLSI) techniques. 

Requirements for military simulators are being driven toward very large gaming areas with relatively 
faithful simulation of natural terrain. These requirements are being pushed as a result of the realiza- 
tion of the potential of full-mission simulators for improving combat readiness. Edge or planar surface 
techniques in computer-generated imagery are not well -suited for representing most natural terrain. Also, 
digitized topographical data from sources such as the U.S. Defense Mapping Agency seem to be the only fea- 
sible basis for very large (millions of square miles) gaming areas. Current edge-based systems utilizing 
this data source depend on a semiautomatic transformation program to convert the digital topographical data 
to an edge data base. The requirements for more efficient ways of representing natural terrain over large 
gaming areas indicate the need to develop nonedge-based scene generation techniques. Such techniques would 
probably use nonlinear surface generation concepts and a direct means for utilizing digitized topographical 
data. 


Computer image generation is rapidly becoming the primary method of scene generation for most simulator 
applications. The final step in computer image generation systems is the conversion of a digital representa- 
tion of the image into an analog television video format. As visual display resolution and detail presented 
per display channel increases, the required band-pass of the video chain increases. However, there are prob- 
lems associated with high-band-pass video systems. Therefore, a display that could use digital information 
directly from the computer image generator would be beneficial. Such displays could take the form of large 
arrays of display elements or some type of matrix-addressed light-valve projector. In view of the many 
advantages offered, the development of such direct-access displays seems worthy of support. 

So far, we have recommended research that needs to be accomplished to: (1) identify the visual cues in 

particular flying tasks, (2) relate those cues to characteristics of the visual simulation system, and 
(3) improve visual simulation technology. One final recommendation pertains to a deficiency that is more 
directly related to the primary purpose of this report. In this report, we have suggested the characteristics 
of the visual simulation system that we believe are necessary to identify its performance. As discussed 
throughout the report and summarized in Section 8, the applicability to visual simulation systems of some of 
these parameters, such as luminance transition density and static MTFA, has not been demonstrated. Moreover, 
we have pointed out the difficulties in developing practical techniques of measuring certain other character- 
istics, such as some of the temporal properties. A research and development effort is needed to identify a 
universally accepted set of measures (perhaps, the set we have proposed) and standardized measurement pro- 
cedures whereby visual simulation systems can be evaluated and compared. In this regard, the main deficiency 
in current techniques is the inability simply to measure the complete visual simulation system from scene 
generation through to pilot display. A study needs to be made of the procedures suggested in Section 4.4 
and alternatives sought. 

In summary, research should be directed (1) to ensure that the existing simulator systems are used to 
(but not beyond) their full capabilities, and (2) to extend the technology to increase the areas where flight 
simulation has adequate fidelity. 
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8. CONCLUDING REMARKS 

This report has described many visual simulation system characteristics, their measurement, and their 
effect or usefulness in visual simulation. When all of these - including field of view, mapping factors, 
scene content, luminance, contrast, resolution, color, noise, excursion limits, time lags, refresh rate, 
and update rate - are considered, the user is faced with a bewildering array of characteristics, many of 
which interact in a complex manner. Furthermore, even if there was a visual system on the market that would 
score high on each of these parameters, it does not necessarily mean that this visual system would fulfill 
all of the requirements that the user is likely to place upon it. Regrettably, we have been unable to give 
anything more than a few words of general advice to potential users of visual simulation systems on how to 
interpret the measured values for these visual parameters and what .trade-off s can be made if the selected 
visual simulation system does not score equally in all parameters. 

Under the category of spatial properties, we have pointed out that scene content is an important measure 
of the quality of the visual simulation system. However, there is no standardized metric for scene content. 

A suggestion is presented for a possible measure but it obviously needs further evaluation. 

In the resolution section, it was shown how MTFA has been found to be a reliable measure for predicting 
performance of certain visual tasks. This metric includes the effect of luminance, contrast, resolution, and 
noise. Providing the noise is not too objectionable, it appears that MTFA might even replace these four 
parameters in a visual simulator system specification. For certain specific tasks, the individual parameters 
may have to be considered but, as an overall measure of system performance, MTFA could be more useful than 
the Individual parameters. 

Although the MTFA has appeal as a measure of overall image quality, it has certain limitations. High 
luminance increases visual acuity and lowers the threshold detectability curve but has little effect on the 
MTFA of systems that are limited to displaying spatial frequencies lower than about 0.2 line pairs per 
arcminute and operate at pho topic levels. Low signal -to- noise ratio will increase the threshold detectability 
curve, particularly at the higher spatial frequencies. High contrast raises the MTF curve at low spatial fre- 
quencies (for a nonnormal ized curve) but very high contrast (i.e,, greater than 50:1) has very little effect 
on the MTFA. Color cannot be included in the MTFA metric, although it invariably seems to lower the MTF of 
a system. 

The Importance of measuring the MTFA under dynamic conditions for application to simulators was men- 
tioned. The static measure of MTFA may not be a useful indicator of the visual simulation system performance 
if, for example, the system contains a component such as a laggy TV camera tube. Because the motion perceived 
in a television display is apparent rather than real, a technique for direct measurement of dynamic MTF is not 
obvious. The desirability of using a metric such as dynamic MTFA seems to be apparent. However, a certain 
amount of research is necessary to establish its validity and applicability as a performance metric and to 
develop standard techniques for its measurement that are relevant to visual simulator systems. Until a 
satisfactory method of measuring MTF under dynamic conditions is found, the dynamic visual -acuity metric 
described in the resolution section may be preferable to static MTFA values. 

As mentioned several times throughout this report, the value of color relative to research or training 
flight simulators has not been well established. If the user can afford to buy full color and if the MTFA 
or any other important parameters are not significantly degraded by the inclusion of color, the choice is 
clear. So many pilots seem to think that color is necessary or desirable, that color should probably be 
considered to be of prime importance until proved otherwise. It is also important to realize that color is 
not merely a property of daylight illumination levels. When pilots are shown typical CGI scenes of an air- 
port at dusk illumination levels, they invariably prefer those generated in full color to those generated 
on two-color penetration CRTs, as long as the reduction in resolution is not too great. 

Certainly the most difficult section of this report to write was the one concerned with the temporal 
properties (Sec. 4). It is apparent to the reader that this subject did not lend itself to the same organi- 
zational structure as did the chapters on spatial and energy properties. It was also difficult to address 
temporal properties in general terms when they are applied to camera -model or model-board systems, CGI sys- 
tems, and film systems. A major difficulty in determining the temporal properties of a simulator visual 
display is the impracticability of making all measurements at the eyepoint. 

Given that such techniques are not possible, a procedure that separates the scene generator from the 
video link appears to be required. Even then, the variety of visual systems available may render a standard- 
ized procedure inappropriate. Nevertheless, since the scene generator contains within it the full definition 
of the movement of the aircraft relative to the scene, a stereotyped set of parameters can be, and has been, 
identified, even though the manner of measuring the values of the parameters may vary according to the system. 
In general (though not invariably), a distinction can be drawn in the manner of measurement between those 
systems in which the viewpoint is manipulated by servo controls (for example, model -board and shadowgraph) 
and those depending entirely on computation techniques (i.e., CGI). In the latter case, while it is possible 
to inject a series of sine-wave input commands (or step inputs) and measure the output at the digital-to- 
analog converter, it will often be more convenient to identify the characteristic parameters of the system 
(transport lag, word length, etc.) and deduce from these the values of the parameters as defined in the 
measurement sections of the chapter on temporal properties. 

Measurements of the. scene generator based on the procedures identified in Section 4.2 should give an 
accurate and consistent definition of characteristics, readily comparable between systems. However, they 
do exclude any compensation techniques for improving performance except where these are inextricably bound 
up with the control system (for example, velocity-controlled model -board systems). Thus, algorithms for 
the alleviation of transport lag in CGI systems, for example, are excluded and cannot readily be encompassed 
because they depend on the update rate of the host (aircraft) computer and the maximimi frequency to be 
attained. 

The method for determining the temporal characteristics of the video link is much less well-defined. 

Those characteristics that vary in a coherent fashion with frequency are generally only consciously 
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detectable at frequencies very much higher than those associated with aircraft motion. However, the video 
link produces many effects that are not representative of the real world and are either frequency or veloc- 
ity dependent, all well within the normal operational envelopes of aircraft. Most of these effects are 
associated with CGI systems, where the discrete nature of the picture-data generation adds to the discrete 
nature of the picture-display generation; there is an interaction between the two and there is no TV camera 
to serve as a smoothing filter. 

Some of the temporal characteristics of the video link are rather general phenomena associated with the 
behavior of the eye and are not necessarily specific to a given system. For example, the perception of 
flicker is a function of the refresh rate and the luminance of the display. However, it also depends on the 
phosphor and how the display is viewed, that is, either foveally or peripherally. Various other phenomena, 
such as doubling of lights, are predictable as functions of the update rate and the angular velocity of the 
lights. An extensive discussion of these and of less predictable phenomena given in the report serves as a 
checklist against which to evaluate the temporal characteristics of the visual simulation system. This dis- 
cussion indicates that quite lengthy exercising of the system is necessary to insure the identification of 
all potential phenomena. 

Several temporal properties of a special nature have not been considered either because there is no 
simple way of measuring them, unless a technique involving the complete system is available, or because they 
are of a special nature. For example, in model -board systems, some elements are on the video side of the 
feedback device used for measuring the output of the scene generator, but are not within the video link. 
Specifically, these include the mechanical connections to the optical elements in the probe and the structure 
between the camera and feedback sensors on the translations. In addition, a false dynamic response is 
obtained by inaccurate alignment of the optical axis with the Image generated on the camera tube. Such 
alignment is achieved statically at a number of discrete orientations but would not necessarily be identified 
from measurement of “spatial properties." There are also some other servocontrolled elements in the camera/ 
model-board systems - for example, cloud/visibility and focus/Scheimpflug optics. Except for the Scheimpflug 
optics, these elements are slow responding, or their accuracies are not of great concern, or the setting 
depends on the task, or they give an inadequate representation for precise results during trials (e.g., 
research into low- visibility landings). 

Finally, a description has been given of several experimental results on the characteristics of human 
visual perception. From this it is clear that the generally discrete nature of the visual display for 
simulators may lead to misinterpretation of the temporal properties of the simulated scene, and be unrep- 
resentative of the true situation. Much of the evidence is inadequate to explain the phenomena fully and 
particular attention needs to be paid to the perception of velocity. 

It is worth noting in conclusion that the way in which a simulator is used can be as Important to its 
effectiveness as are its motion or visual simulation capabilities. With respect to training, the simulator 
is a valuable tool for use in part of a total training program. Reductions In training time in the simulator 
and in the aircraft have resulted from improving the methods of training and from making the most effective 
use of the simulator in full recognition of its limitations. Similarly, in research and development, the 
flight simulator is an effective tool, but clever design and execution of the simulation experiments within 
the capabilities of the simulator are essential. 
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APPENDIX A 
FIELD-OF-VIEW PLOTS 


Two methods of depicting a large field of view are commonly employed: the simple uniform rectangular 

grid and the projection due to Hammer (which is sometimes ascribed to Aitoff). The uniform rectangular grid 
plot needs no further explanation. The Harrmer equal-area projection is composed of curved lines and consti- 
tutes a flat projection of a complete sphere of radius R. The vertical and horizontal coordinates of a 
particular location on the sphere's surface are given by the following functions of elevation angle {lati- 
tude} and azimuth angle (longitude): 

I = 2 cos 0/2[l - cos(y/2}] 

w- • m‘ 

b -cos e 

^ /?(/l - cos e - sin e) 

^ - /2(l - cos~ey 

where 

0 = elevation angle (latitude) 

Y = azimuth angle (longitude) 

R = sphere radius 

The above equations yield the coordinates in the upper-right quadrant of the projection. The remaining por- 
tions may be inferred by its symmetry. An example of the Hammer projection^® is shown in Figure 65. 

In the following figures, the fields of view of several aircraft are shown. Figure 66 is the binocular 
vision plot for a typical strike aircraft, and Figure 67 is the vision envelope for the same aircraft with 
pilot head movement. The remaining examples (Figs. 68 through 73) are the monocular fields of view; Fig- 
ures 68 through 70 are on Hammer's projection, and Figures 71 through 73 are on rectangular grid plots. 

In Figures 65 through 73, the location of the runway threshold center is also shown for various approach 
profiles. In these figures, the simulator fields of view are also shown. 



Fig. 65 Hanmer's equal -area projection of a sphere 


Fig. 66 Military strike aircraft (Panavia Tornado) 
binocular vision plot 


^®Hairmer projection graph paper is available from the U.S. Department of Conmerce, Environmental Science 
Service Administration, Washington, D.C. 
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LEFT EYE WITH 

head movement 



RIGHT EYE WITH 
HEAD MOVEMENT 



BLIND AREA 


Fig. 67 Military strike aircraft (Panavia Tornado) 
vision envelope with head movement 




Fig. 69 Small jet transport (Falcon 20C) 



Fig. 70 High-wing light aircraft (Cessna 150M) 


RUNWAY THRESHOLD 



TOTAL FIELD OF VIEW 
WITH HEAD MOVEMENT 




Fig. 71 Captain's vision envelope: A300 


VERTICAL DEFLECTION, deg 






RUNWAY THRESHOLD 



TOTAL FIELD OF VIEW 
WITH HEAD MOVEMENT 



Fig. 72 Captain's vision envelope: 747 



TOTAL FIELD OF VIEW 
^ WITH HEAD MOVEMENT 


HORIZONTAL DEFLECTION, dog 



Fig. 73 Captain's vision envelope: DC«10 


VERTICAL DEFLECTION, dog VERTICAL DEFLECTION, deg 
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APPENDIX B 

PHOTOMETRIC CONVERSION FACTORS 


QUANTITY 

MULTIPLY 

BY 


TO OBTAIN 

Luminous flux, F 

Lumens 

7,9578 

X 10 "’ 

Spherical candlepower 


lumens at 555 um 

1.4706 

X 10*’ 

Watts 


Spherical candlepower at 555 pm 

1.848 X 

10-2 

Watts 


Watts 

1.0 


Joules/ sec 

Luminous intensity, I 

Candle 

1,0 


Candela 


Candela 

1.0 


Candlepower 


Candela 

1.0 


Lumens/steradian 

Illuminance, E 

Lumens/cm^ 

1.0 


Phot 


Lumens/cm^ 

9.29 X 

102 

Foot-candela 


Lumens/m^ 

1,0 


Lux 


Lux 

1.0 


Meter-candela 


Foot-candela 

1.0 


Lumens/ft2 

Luminance, B 

Apostilb 

1.0 X 10 -“ 

Lamberts 


Candela/ft^ 

3.142 


Foot-lamberts 


Candela/ft^ 

3.382 X 

10 - ’ 

Lamberts 


Candela/ft^ 

1.076 X 

10 "^ 

Candela/cm^ * stilb 


Candela/cm^ 

1.0 


Stilb 


Candela/m^ 

1.0 


Nit 


Candela/m^ 

2.919 X 

10 “^ 

Foot-lamberts 


Candela/m^ 

3.142 X 

10 " 

Lamberts 


Foot-lamberts 

1.076 X 

10-3 

Lamberts 


Stilb 

1.0 


Candela/cm* 


Nit 

1.0 


Candela/m* 
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APPENDIX C 

GENERATION OF SINUSOIDAL INPUT SIGNALS AND ANALYSIS OF THE OUTPUT SIGNALS 


INTRODUCTION 

The generation of the visual system input signals and the sampling and analysis of the output signals 
must be standardized in order that different systems may be compared with each other. For convenience, as 
is explained in the main text, sinusoidal input signals are used in most of the prescribed tests. The pro- 
cedures prescribed are to be followed in detail in order to assure mutually comparable results. 

Engineering parameters that contribute to visual system dynamic characteristics have to be defined. 
These characteristics are: 

1. Excursion limits 

2. Describing function 

3. Linearity 

4. Noise 

5. Hysteresis 

6. Threshold 

Characteristics (l)-{5) are measured by applying a sinusoidal input signal to the system. A single 
sinusoid (over a range of frequencies and amplitudes) is used, for this signal is in reasonable accordance 
with signals for rotation during simulation, and because a single input signal improves the data analysis. 
The linear translations are measured with a constant speed input signal as well. 

After discrete Fourier transformation, the measured output signal is split up into fundamental and 
position signal noise. 

If necessary this noise can be subdivided into lower harmonics, higher harmonics, and stochastic 
residue. This subdivision is useful if it is desired, for example, to locate the source of excessive noise. 



DISCRIBING FUNCTION 
LINEARITY 


LOW-FREQUENCY NOISE 
NONLINEARITY 


A limited set of measurement runs thus enables the identification if describing function, linearity, and 
noise. 


The excursion limits are divided into system limits and operational limits . System limits are the 
absolute boundaries of displacement and its time derivatives. The operational limits, that is, the usable 
excursion limits, are measured by varying the amplitude at certain specified frequencies until the noise 
ratio exceeds a standardized level. 

The describing function is measured at an amplitude of 10% of the system limits at recommended fre- 
quencies. The measurement to determine the operational limits is also used to calculate linearity (at 
0.5 Hz for rotation and possibly at a lower frequency for linear translation). A very low-frequency input 
is used to measure hysteresis , and the response to a step input is used to define threshold . 
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SIGNAL GENERATION DATA SAMPLING 

Figure 74 shows the general signal flow for 
signal generation and data logging. The prescribed 
sampling interval is At = 0.01 sec, which results in 
a Nyquist frequency of 50 Hz (314 rad/sec). Since 
the frequency content of interest extends up to 35 Hz 
(220 rad/sec), the contamination of the measured 
signals by crosstalk from ac power supplies must be 
eliminated in practice. The measured signals must 
be analog-prefil tered in case they contain power 
above 65 Hz (408 rad/sec) in order to avoid aliasing 
effects below the upper-frequency limit of 35 Hz. 

The design of the presampling filter will be a com- 
promise between overall accuracy and complexity. The 
only stringent requirement placed on the filter is 
the reduction of any signal power above 65 Hz to an 
insignificant level prior to sampling. The distor- 
tion due to presampling filtering in the frequency 
range of interest, that is, lower than 35 Hz 
(220 rad/ sec) can be corrected during discrete 
signal processing. The order of the low-pass pre- 
sampling filter influences the necessary correction. 
As the order increases, the amount of necessary 
correction decreases, and a corresponding increase 
in the overall accuracy results. The run length will 
be 1024 (2^°) samples at 100 samples per second, 
resulting in a frequency resolution of 
(1/10.24) Hz = 0.0977 Hz (0.6136 rad/sec). 


rBlGITAL COMPUTER SYSTEI^ 



I 1 

Fig. 74 Signal generation and data logging 


The input signals are generated in the time domain using the following expression: 

X(n*At) = a sin ^ ^ (n*At)j . n = 0, 1, 2 N - 1 

where 

a = amplitude of the input signal 
2it k^- 

- frequency of the input signal (rad/sec) 

k^ = integer' number of periods of the input signal within the measurement length 
of N samples (N = 1024) 


By computing the time series of the input signals, using the expression given above, an integer number of 
periods within the sampling period is guaranteed. Before starting the sampling procedure, the response of 
the system must be stationary; there must be no influence from transients due to improper initial conditions. 
The moments at which the inputs to the visual system are updated and of the instants in time of the signal 
sampling must be carefully synchronized (preferably controlled by the same clock). The sample time At 
must be accurately held constant. 


ANALYSIS OF THE OUTPUT SIGNALS 

The set of characteristics computed from the output signals are related to the standard set of mutually 
independent output signals. Let the standard set of output signals be represented by: 




*Xs(t)" 

ys(t) 

Zs(t) 

0s(t) 


The standard output signals Xo(t) can be computed from a suitable set of transducer signals Xjn(t) 
according to: 

io(t) = [Tm]xm{t) 


(Cl) 
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The matrix T„, depends on the location and type of the transducers. The measured signals in digitized 
form are distorted by the characteristics Htr(jw) of the transducer (including presampling filter). Sig- 
nals not compensated for the transducer dynamics are indicated below by an apostrophe [x,ii(t)]. In Figure 75, 
a block diagram of the basic signal processing is shown. The first part of the processing embraces the 
calculation of the discrete Fourier transform (DFT) of the standard output signals, symbolically written as: 

Xo(k) « DFT[Xo(n)] , k = 0. 1, 2, 3, .... N - 1 (C2) 

In the block diagram of Figure 76, this part of the processing is presented in detail. If the transformation 
matrix T^ describes a linear transformation, the computation of XQ{k) can easily be reduced, as will be 
shown later on (Eq. (C6)). In general, however, the calculation proceeds as follows. 

First, the sampled time sequence X|j,(n) are corrected for the transducer characteristics (Htr). For 
reasons of computational efficiency, the operation can best be performed in the frequency domain. The DFT 
of ^{n) may be computed via a fast Fourier transform (FFT) routine yielding 

Xtn(k) = DFT[x^(n)] (C3) 


Using 

x„,(k) - X^OO/Htr (j 

the transducer dynamics are completely eliminated. Applying the inverse discrete Fourier transform (IDFT) 
yields the corrected measured signal sequences in the time domain: 

x„(n) - IDFT{yk)} (C4) 


From these time sequences for Xn,(n), the standard output signals XQ(n) can be obtained, using Eq. (Cl). 
Finally, the output signals XQln) are transformed to the frequency domain by computing the OFT of these 
sequences. 



Fig, 75 Data analysis Fig. 76 Computation of discrete Fourier transform; 

of the standard signals 
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The complete procedure described above may be combined into: 


Xo(k) = DFT 



(C5) 


It can easily be seen that if Tm describes a linear transformation, Eq. (C5) can be simplified to: 


Xo(k) 


D»^([Tm] • !^(n)[ 


(C6) 


The power per frequency interval a^(k) for the different signals can be computed from their DR 
coefficients: 


oMk) = Clx(k)|" (C7) 

Where C is a constant which depends on the definition of the applied DFT, as will subsequently be shown. 

Since extensive use of DFT concepts is made deriving the results presented in this report, and since 
the definitions embodied in these DFT concepts are presented in the literature in different ways, clarity 
in formulation is demanded in order to avoid confusion. We herein define the DFT of a sequence of N 
samples as 

X(k) = DFT{x(n)} = a x(n) exp^-j k = 0, 1. 2 N - 1 (C8) 

n=o 

The corresponding IDR is 

N-i 

x{n) = IDFT{X(k)} = f ^ X(k) exp^j , n = 0. I, 2, .... N - 1 (C9) 

k=o 


Where a is an arbitrary constant. For instance, in Reference 72, a definition has been used specifying 
a equal to 1/N; in Reference 73, a is taken equal to 1. As both X(k) and x(n) are periodic with 
period N, according to Eqs. (C8) and (C9), shifted versions of the definition outside the ranges 
0, 1, . . ,, N - 1 can be found in the literature as well. 

It can be shown that an estimate of the power per frequency interval is given by the following 
expression when the factor a is retained (Ref. 72): 

o"(k) = ^ [lX(k)|^ + |X(N - k)|^] . k = 1. 2 (^ - l) 

Since |X(k)| = |X(N - k)| and |X(k)| = |X*(k)l, this relation can be simplified to: 

a N 

oHo) 

„ Cl ” 

Taking distinct combinations of the frequency intervals, the power of the separate components can be com- 
puted as is done in Section 4. 2. 3. 2 in the main text. 

The reader familiar with spectral analysis of sampled time sequences will note that the so-called 
unsmoothed spectral estimate has been used to estimate the power per frequency interval in Eqs. (CIO) and 
(Cll). If stochastic signals are present, Eq. (Cll) provides an inconsistent estimate of the power per 
frequency interval (Refs. 73, 74). However, the averaging of sequential estimates or the application of 
an appropriate window can provide a consistent estimate. The types of signals considered in this report 
are basically deterministic periodic signals contaminated by a relatively small stochastic component. In 
this case, the variance of the estimate of the power in a frequency interval containing the contributions 
Of the deterministic part of the signal is relatively small. In the final presentation of the results, 
only figures obtained by summing over mutually orthogonal stochastic power contributions are considered. 

This results in small variances for the estimated sums. The only acceptable smoothing procedure for the 
analysis involved is averaging over sequential estimates. This is sometimes referred to as the Barlett 
procedure (Ref. 73). This procedure consists of repeating the measurements and the basic analysis. Applica- 
tion of any other known smoothing procedure is unacceptable due to the bias introduced, which results from 
leadage. The leadage involved will violate the funda^ntals of the analysis presented. If the prescribed 
run length and sample interval are used without smoothing, the analysis will usually result in sufficiently 
consistent estimates. Only in those cases in which the signal contains a relatively large stochastic com- 
ponent will the consistency be poor and smoothing become necessary. In such a case, the question can be 
raised whether it will be worthwhile to produce accurate figures for inaccurate systems. 


(CIO) 


(Cll) 
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CHARACTERISTICS OF FLIGHT SIMULATOR VISUAL SYSTEMS 


SUMMARY . 

Out-of“the-window visual simulation is a formidable challenge because of the fantastic performance 
capabilities of the human eye. It is, of course, totally impractical to set the requirements of a visual 
simulation system to match the performance of human eyes. Those who set requirements and determine speci- 
fications for simulator visual systems find themselves in a quandary. The technology is inadequate to pro- 
vide the "ideal" system and our understanding of how a human uses the visual information in a simulator is 
insufficient to provide clear guidelines on how to make the necessary trade-offs. The continuing task, 
then, is to define the design characteristics that may affect perception of physiological responses, to 
establish the relative importance of the corresponding visual and physiological effects, and to understand 
their relationship with the physical continuums of the displays that can now be generated. 

This report addresses only a very small part of the total problem. The objective here was to identify 
and define those physical parameters of the flight simulator visual system that characterize the system and 
determine its fidelity. The desire was to establish the physical measures of image quality that are describ- 
able in objective terms. It is convenient to discuss the characteristics of visual simulation systems in 
terms of the three basic categories of spatial, energy, and temporal properties corresponding to the three 
fundamental quantities of length, mass, and time. This report discusses the characteristics of visual simu- 
lation systems within these categories, and for each of the parameters there is a description of its effect, 
a definition of its appropriate units or descriptors, a discussion of methods of measurement, and a discus- 
sion of its use or importance to image quality. There is also a presentation of the experience of the 
Working Group members regarding the importance of these parameters in accomplishing a given visual task 
under given conditions. The final chapters of this report present projections of future trends and recom- 
mendations for research. 

This report was prepared by a Working Group sponsored by the Flight Mechanics Panel of AGARD. 
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